Posting in the Magento forums has been disabled pending the implementation of a new and improved forum solution which should better serve the community.

For new questions please post at magento.stackexchange.com, the community-run support site for the Magento community. We will be providing updates on the new forum solution soon. For questions or concerns please email community@magento.com.

Magento Forum

the quest for an efficient product import script
 
warehousepoint
Member
 
Total Posts:  32
Joined:  2011-11-25
 

is there a way we can update & create new items without clearing the catalog first?

We have csv files for our inventory that get dumped to our server.  right now, we clear the catalog and run the magmi commands to create and then reindex.  is there a way we can update using our csv files but if an item is not listed in the csv anymore, it has its qty set to 0 and showed as out of stock?

that way there is no downtime when we import and have no products appearing.  please help, thank you.

 
Magento Community Magento Community
Magento Community
Magento Community
 
Bary
Jr. Member
 
Total Posts:  12
Joined:  2010-11-18
 

Let me understand: you have csv, you clear catalogue and into empty catalogue you import your csv. Now you have a new csv you want to import so you clear catalogue and import new one?

Why do not you compare those two csv files? Find out what sku inside older one is not in newer one. You have a 3 possibilities now. Create csv just with those skus and columns:
1. is_in_stock = 0 and qty = whatever (if you do not want backorders - magento show these products “Not in stock") or
2. is_in_stock = 1 a qty = 0 (if you want backorders - carefull!! Magento show this products as “In stock” even it is logically not because qty=0. So you must edit your template files to show those product “Not in stock” but backorders will be fully functional - megento thinks the product is in stock, remember smile ) or
3. magmi:delete = 1 with product deleter plugin ON with your profile (this will delete those products form magento)

 
Magento Community Magento Community
Magento Community
Magento Community
 
shmk
Sr. Member
 
Total Posts:  297
Joined:  2009-10-28
 

Quick question: is it safe to execute an import without reindexing, execute in sequence another import (maybe with products edited on the first import) and only then execute a reindexing? Or at the end of every import a reindex it’s needed to avoid error in the next import?

 
Magento Community Magento Community
Magento Community
Magento Community
 
warehousepoint
Member
 
Total Posts:  32
Joined:  2011-11-25
 

when we run our import now via our con, we get the below error.

php public_html/magmi/cli/magmi.cli.php -mode=update -profile=allproducts

Warning: require_once(../inc/magmi_defs.php): failed to open stream: No such file or directory in /home/cdistore/public_html/magmi/cli/magmi.cli.php on line 12

Fatal error: require_once(): Failed opening required ‘../inc/magmi_defs.php’ (include_path=’.:/usr/lib/php:/usr/local/lib/php’) in /home/cdistore/public_html/magmi/cli/magmi.cli.php on line 12

please help.

 
Magento Community Magento Community
Magento Community
Magento Community
 
dweeves
Enthusiast
 
Total Posts:  877
Joined:  2010-06-26
FRANCE
 

@shmk

this is perfectly safe & a good way to optimize “global” import time.

Magmi magento reindexer calls magento reindexing script that performs full reindexing on each specified index , so it can be called once for all at the end of all magmi imports.

 
Magento Community Magento Community
Magento Community
Magento Community
 
shmk
Sr. Member
 
Total Posts:  297
Joined:  2009-10-28
 
dweeves - 12 March 2013 11:41 PM

@shmk

this is perfectly safe & a good way to optimize “global” import time.

Magmi magento reindexer calls magento reindexing script that performs full reindexing on each specified index , so it can be called once for all at the end of all magmi imports.

Perfect!

 
Magento Community Magento Community
Magento Community
Magento Community
 
mikesimaska
Jr. Member
 
Total Posts:  2
Joined:  2013-02-13
 

I have about 3500 products to import into a new magento store.

The current catalog is sitting in a mysql db.  I generate a dataflow-compatible CSV from this table using a small php script. 

Using Import All Products brings this test CSV into my products catalog successfully. 

Of course it’s terribly slow and does not import images. 

So I’ve turned to MAGMI for help.

But MAGMI is choking on the small test file.

I’ve read the wiki and can’t figure out what’s going wrong.

Found 80 records, took 0.00394010543823 sec
CSV Datasource v1.3 - No BOM detected, assuming File is UTF8 without BOM…
CSV Datasource v1.3 - 78 CSV headers columns found
Standard Attribute Import v1.0.5 - Newly created items will have default values for columns:store,type,status,visibility,page_layout
Ajusted processed columns:83
80 warning(s) found Hide Details
CSV Datasource v1.3 - warning: line 1 , wrong column number : 79 found over 78, line skipped
CSV Datasource v1.3 - warning: line 2 , wrong column number : 79 found over 78, line skipped
CSV Datasource v1.3 - warning: line 3 , wrong column number : 79 found over 78, line skipped
CSV Datasource v1.3 - warning: line 4 , wrong column number : 79 found over 78, line skipped
CSV Datasource v1.3 - warning: line 5 , wrong column number : 79 found over 78, line skipped

There’s an error for each line. 

Help?

*UPDATE*

searching this thread led me changing the delimiter character to “|” which cleared the first hurdle.  Now to figure out images....

 
Magento Community Magento Community
Magento Community
Magento Community
 
dweeves
Enthusiast
 
Total Posts:  877
Joined:  2010-06-26
FRANCE
 

@mikesimaka

In order to import images, you MUST install the “extra plugin package” & configure image processor plugin.

 
Magento Community Magento Community
Magento Community
Magento Community
 
vladino
Jr. Member
 
Total Posts:  8
Joined:  2009-02-27
 

Hello, I use Magmi and its a great tool but I cant help myself with right syntax for the Value Replacer.
Please should you help me with example?
I cant find it and I was searching a lot. (Maybe Im blind)

First problem - simple change
- my column in data source is “CATEGORY”
- value in “CATEGORY” is “Books”
- value for magento column “category_id” is “62”

Second problem - calculation
- my column in data source is “WEIGHTINGRAMS”
- value in “WEIGHTINGRAMS” is “300”
- value for magento column “weight” I want to be “0,3”

[ValueReplacerItemProcessor]
VREP
:columnlist "???"
VREP:???= "???"

Thanks

 
Magento Community Magento Community
Magento Community
Magento Community
 
dweeves
Enthusiast
 
Total Posts:  877
Joined:  2010-06-26
FRANCE
 
vladino - 10 April 2013 05:18 AM

Hello, I use Magmi and its a great tool but I cant help myself with right syntax for the Value Replacer.
Please should you help me with example?
I cant find it and I was searching a lot. (Maybe Im blind)

First problem - simple change
- my column in data source is “CATEGORY”
- value in “CATEGORY” is “Books”
- value for magento column “category_id” is “62”

Second problem - calculation
- my column in data source is “WEIGHTINGRAMS”
- value in “WEIGHTINGRAMS” is “300”
- value for magento column “weight” I want to be “0,3”

[ValueReplacerItemProcessor]
VREP
:columnlist "???"
VREP:???= "???"

Thanks

First i would advise you to use the web UI for configuring the value replacer instead of directly trying to fill the .ini

For your CATEGORY problem, i’d avise to use “category importer/creator plugin” in order to be able to match magento textual values for categories.

if you have only books in your file, then you can force “category_ids” value through value replacer as well:

so for both problem, using UI , in “values to replace” field put:

category_ids,weight (ie : magmi/magento target column names)

Then dynamic fields will appear:

replacing value for category_ids : 62
replacing value for weight : {{ {item.WEIGHTINGRAMS}/1000 }} (with the {{ }} around to use “extended syntax”

Save the profile, then the final values will appear in the .ini

 
Magento Community Magento Community
Magento Community
Magento Community
 
lake_tuna
Member
 
Total Posts:  54
Joined:  2011-03-07
 

dweeves,

Any plans to update Magmi for Enterprise 1.13, which was just released this week?

 
Magento Community Magento Community
Magento Community
Magento Community
 
lake_tuna
Member
 
Total Posts:  54
Joined:  2011-03-07
 

I’m looking at this line below in magmi_productimportengine.php..

$this->datasource->getNextRecord()

Does anyone know where I can find the code for this function, getNextRecord()?

Also, I’ve looked at most of the PHP files, but I’m not sure how Magmi creates all of the data necessary for the tables with prefix “catalog_product_entity” (_varchar, _decimal, etc.). I’m seeing that the function createProduct() sets up data only for the main table, catalog_product_entity, but where can I find the code where it set up data, for example, catalog_product_entity_varchar?

 
Magento Community Magento Community
Magento Community
Magento Community
 
dweeves
Enthusiast
 
Total Posts:  877
Joined:  2010-06-26
FRANCE
 

@lake_tuna

- the processing of values to store in eav type dependent table is done through createAttributes method in productImportEngine.
- the getNextRecord() method can be found either in plugins/base/datasource/csv/magmi_csvdatasource or plugins/base/datasource/genericsql/sql_datasource.php

 
Magento Community Magento Community
Magento Community
Magento Community
 
kek0
Jr. Member
 
Avatar
Total Posts:  28
Joined:  2010-03-06
 

Hi! I am trying to use Column mapper v0.0.3 for change two columns:
SKU -> sku
STOCK -> qty
I was set in the magmi configuration in the first line SKU, STOCK and appears two new boxes for add the word for replace each word.

In my case only work the first one, if the first is SKU, and second STOCK, only map sku
If the first is STOCK, and second SKU, appears and error that don’t exist the sku column, for the same reason.

Is a bug, or maybe I was setup something wrong?

Using last magmi version 0.7.18

Thanks! smile

 
Magento Community Magento Community
Magento Community
Magento Community
 
vladino
Jr. Member
 
Total Posts:  8
Joined:  2009-02-27
 
dweeves - 11 April 2013 04:39 AM

vladino - 10 April 2013 05:18 AM
Hello, I use Magmi and its a great tool but I cant help myself with right syntax for the Value Replacer.
Please should you help me with example?
I cant find it and I was searching a lot. (Maybe Im blind)

First problem - simple change
- my column in data source is “CATEGORY”
- value in “CATEGORY” is “Books”
- value for magento column “category_id” is “62”

Second problem - calculation
- my column in data source is “WEIGHTINGRAMS”
- value in “WEIGHTINGRAMS” is “300”
- value for magento column “weight” I want to be “0,3”

[ValueReplacerItemProcessor]
VREP
:columnlist "???"
VREP:???= "???"

Thanks

First i would advise you to use the web UI for configuring the value replacer instead of directly trying to fill the .ini

For your CATEGORY problem, i’d avise to use “category importer/creator plugin” in order to be able to match magento textual values for categories.

if you have only books in your file, then you can force “category_ids” value through value replacer as well:

so for both problem, using UI , in “values to replace” field put:

category_ids,weight (ie : magmi/magento target column names)

Then dynamic fields will appear:

replacing value for category_ids : 62
replacing value for weight : {{ {item.WEIGHTINGRAMS}/1000 }} (with the {{ }} around to use “extended syntax”

Save the profile, then the final values will appear in the .ini

Thanks, but my problem still persist.

1. CATEGORIES - I use csv acording to instruction from magmi wiki. I have just one category for each product. Source column is “CATEGORY”
In my file kategorie.csv are all categories in order: Source Name Of Category;Target Name Of Category
In field “Replaced attributes” i have “categories”
New value for categories=

{{ ValueRemapper::use_csv('http://www.mysite.sk/kategorie.csv')->mapmulti({item.CATEGORY},'/'}}
On the fly category creator/importer v0.2.3 - on
Column mapper v0.0.3 - I dont use it for this column (Should I have to use it?)
With this options - magmi create categories from source column without remaping.

2. Weight - In datasource I have values with coma (0,3).
Without calculating, just column remaping, magmi import value and self change coma to dot (0.3).
But if I use {{ {item.WEIGHTINKILOGRAMS}*1000 }}, imported value is empty.
When I manually change coma to dot in datasource, calculating value is right - 300.

 
Magento Community Magento Community
Magento Community
Magento Community
Magento Community
Magento Community
Back to top