Posting in the Magento forums has been disabled pending the implementation of a new and improved forum solution which should better serve the community.

For new questions please post at magento.stackexchange.com, the community-run support site for the Magento community. We will be providing updates on the new forum solution soon. For questions or concerns please email community@magento.com.

Magento Forum

the quest for an efficient product import script
 
spilla
Jr. Member
 
Total Posts:  8
Joined:  2008-09-06
 
dweeves - 11 July 2013 05:12 AM

@spilla
1 - use latest 0.7.18 (don’t forget latest plugins versions too) or even better, the git version (which is a to be release 0.7.19 with enhancements)
2 - unless explicitely told to (through value replacer) ,magmi does not perform any computations on input values.

Cheers

Dweeves, the update to 0.7.18 did not fix the problem.

It seems that Magmi is adding (or subtracting?) the negative integers onto Magento’s previous SOH instead of replacing it.

For example, an SKU has a SOH of 0 in the DB and in Magento, and one is sold, so the new SOH is -1 in both the DB and Magento.

When Magmi runs the next import, the SKU’s SOH will be -2 in Magento & -1 the DB.

Then the next time the Magmi import runs, Magento’s SOH will be -3, while the DB is still -1.

This continues, and eventually SKU’s can have SOH of -2000!

Are there any fixes in the 0.7.19 version that may address this?

 
Magento Community Magento Community
Magento Community
Magento Community
 
dweeves
Enthusiast
 
Total Posts:  877
Joined:  2010-06-26
FRANCE
 

@spilla : indeed that’s the “native” relative stock feature from magmi. when a stock is set with a “-” or a “+” ,then magmi considers it as relative adjustement from current value.

I’ll update the magmi_productimportengine on git version ASAP to allow disabling of this “automatic relative detection”. (surely by adding relative_qty column , so it can be set by default with value replacer or default value setter plugin for all values without changing input file or source)

 
Magento Community Magento Community
Magento Community
Magento Community
 
xpaz
Jr. Member
 
Total Posts:  4
Joined:  2008-09-24
 

Hi,

I am using Magmi for some weeks now (it’s a great tool, marvelous compared to the standard Magento import). I am using it mainly to update an existing catalog, to make changes in values to a list of products - i.e. load some thousand images and PDFs, flag products as discontinued, or establish product relations (x-sell and related products). However I cannot change the list of grouped products - simples that are listed as part of a grouped product.

This is a sample of the file I use (csv using colon as field separator):

store;websites;attribute_set;sku;grouped_skus;
admin;base;Default;MOD-GM4100;GM4100-WH-910,GM4100-HC-D433,GM4100-HC-433,GM4100-WH-433,GM4100-BK-433,GM4100-BK-D433,GM4130-WH-433K1,GM4130-WH-433K2,GM4130-BK-433K2,GM4130-BK-433K1;

This exact format works for all other fields, but not for this. The plugin “Grouped Item processor v1.1” is enabled, and the import does not report any problem, neither on screen nor in the log file. All products do exist, and I can add them to the grouped products list from the admin without any problem.

What am I missing?

 
Magento Community Magento Community
Magento Community
Magento Community
 
phaimua2000
Jr. Member
 
Total Posts:  1
Joined:  2013-07-25
 

It worked very good.
Thanks you!

 
Magento Community Magento Community
Magento Community
Magento Community
 
gilmr
Jr. Member
 
Total Posts:  10
Joined:  2011-03-01
 

Hi there,
I’m having problems with Custom Options import. When I’ve imported all the products custom options for the first time, it worker perfectly, but now I’m trying to update some of the products, not only those products weren’t updated it erased the others custom options that wasn’t in the csv file. Now I’m trying to import all the products custom options again, but nothing is happening, no changes are made.
For this I’ve made a new csv file with the sku and the the custom options and selected the Magmi Optimizer v1.0.5, Custom Options v0.0.7 and Value Trimmer for select/multiselect v0.0.3.
Please help

 
Magento Community Magento Community
Magento Community
Magento Community
 
ofecrpnr
Jr. Member
 
Total Posts:  1
Joined:  2013-08-01
 

Thank you so much, it saved my whole day!!

 
Magento Community Magento Community
Magento Community
Magento Community
 
phimola3
Jr. Member
 
Total Posts:  1
Joined:  2013-08-06
 

Dweeves, cheers for the reply! I changed the category header as you stated and it worked perfectly everything has imported and appears as it should in admin. With that said however the products are not appearing on frontend unless I select a product and then click save (I don\’t have to change any settings). After doing this, the product appears on the frontend.<a href=’http://tinhay30s.com/tag/clip-sex/’ rel="dofollow" title=’clip sex’>clip sex</a>

 
Magento Community Magento Community
Magento Community
Magento Community
 
phimmoi
Jr. Member
 
Total Posts:  1
Joined:  2013-08-06
 

Không được gắn dấu sao
Magento
Newsletter subscription success - Newsletter subscription success
22:03
Không được gắn dấu sao
Magento
Welcome, phim! - Magento Inc. Dear phim, Welcome to Magento. We have created your Magento account. You can login to
22:03
Không được gắn dấu sao
Registration
SmartViper Registration Confirmation - Hello, please click “activation” link for complete registration. Activate account Thanks,

 
Magento Community Magento Community
Magento Community
Magento Community
 
tinhay30s
Jr. Member
 
Total Posts:  1
Joined:  2013-08-06
 

thanks very good,i like you

 
Magento Community Magento Community
Magento Community
Magento Community
 
dweeves
Enthusiast
 
Total Posts:  877
Joined:  2010-06-26
FRANCE
 

@phimola3

to avoid “saving” in the backend, don’t forget to reindex after magmi importing !!!! (you can do it using magmi magento indexer plugin, choosing which indexes you want to rebuild)

 
Magento Community Magento Community
Magento Community
Magento Community
 
dhecar
Jr. Member
 
Total Posts:  2
Joined:  2011-07-25
 

Using v0.7.19_svn version of MAGMI but I’m getting this error when try to import images:

Image attributes processor v1.0.26 - error creating media/catalog/product/m/f: 2,mkdir(): Permission denied
Image attributes processor v1.0.26 - error creating media/catalog/product/m/f: 2,mkdir(): Permission denied
Image attributes processor v1.0.26 - error creating media/catalog/product/m/f: 2,mkdir(): Permission denied

All permission are ok..

What could be the problem?

 
Magento Community Magento Community
Magento Community
Magento Community
 
BrotherMouzone
Jr. Member
 
Total Posts:  18
Joined:  2010-06-20
 

Edit:  Got it to work. Instead of using “default values setter”, I gave the values in the foreach loop array. Now my import is running in around 10 seconds instead of 10 minutes.

Hi guys.

Do any Magmi expert have advice on making Magmi datapump run faster? I’ve got some installations that run CSV files on import very fast, but now I’m building a site that imports a “huge” 45 megabyte XML-file (about 15 000 products worth of data). I updgraded my server since my import was taking 10 minutes, even with only very few attributes being used on creation.I’m limiting imports to products from a certaint marketer id with Import Limiter with this filter: !marketer::.*1500.* I’ve got Magmi Optimer enabled as well. I’m guessing the code is using to read the XML file is horribly inefficient?

Each item is being read like this:

$file='/var/www/public_html/var/import/downloadedXML.XML';      
$fullxml = new SimpleXMLElement($filenulltrue);  

foreach (
$fullxml->Package as $item){
    $newProductData 
= array(
            
'sku'=>(string)$item->Singledata->sku
                        
'price'=>(real)$item->price_data->price,
            
'marketer'=>(string)$item->Singledata->Marketer["Id"]            
    
);
    
$dp->ingest($newProductData);
    
$newProductData=null;
    unset(
$newProductData);
}
unset($fullxml); 
$dp->endImportSession();

Thanks for any help.

 
Magento Community Magento Community
Magento Community
Magento Community
 
dweeves
Enthusiast
 
Total Posts:  877
Joined:  2010-06-26
FRANCE
 
BrotherMouzone - 08 August 2013 03:52 AM

Edit:  Got it to work. Instead of using “default values setter”, I gave the values in the foreach loop array. Now my import is running in around 10 seconds instead of 10 minutes.

Hi guys.

Do any Magmi expert have advice on making Magmi datapump run faster? I’ve got some installations that run CSV files on import very fast, but now I’m building a site that imports a “huge” 45 megabyte XML-file (about 15 000 products worth of data). I updgraded my server since my import was taking 10 minutes, even with only very few attributes being used on creation.I’m limiting imports to products from a certaint marketer id with Import Limiter with this filter: !marketer::.*1500.* I’ve got Magmi Optimer enabled as well. I’m guessing the code is using to read the XML file is horribly inefficient?

Each item is being read like this:

$file='/var/www/public_html/var/import/downloadedXML.XML';      
$fullxml = new SimpleXMLElement($filenulltrue);  

foreach (
$fullxml->Package as $item){
    $newProductData 
= array(
            
'sku'=>(string)$item->Singledata->sku
                        
'price'=>(real)$item->price_data->price,
            
'marketer'=>(string)$item->Singledata->Marketer["Id"]            
    
);
    
$dp->ingest($newProductData);
    
$newProductData=null;
    unset(
$newProductData);
}
unset($fullxml); 
$dp->endImportSession();

Thanks for any help.

Indeed, loading a big XML file using only DOM parsing is a performance & memory killer.
A good way to deal with big XML files is a mix between SAX parsing & DOM parsing (using XMLReader Object)

some examples of this technique can be found here:
http://stackoverflow.com/questions/1835177/how-to-use-xmlreader-in-php

 
Magento Community Magento Community
Magento Community
Magento Community
 
shanana
Jr. Member
 
Total Posts:  11
Joined:  2009-04-26
 

hi dweeves,

I don’t see anything in the docs about setting a sort_order for images. Is that something we can actually do in magmi?

Thanks!

 
Magento Community Magento Community
Magento Community
Magento Community
 
spilla
Jr. Member
 
Total Posts:  8
Joined:  2008-09-06
 

Dweeves, what file permissions should I be giving progress.txt to use over both the CLI (magmi.cli.php) and web UI?

I’m running the v0.7.19_svn version (from Git).

When I give progress.txt permissions of 0777, the CLI & web UI work fine, but as soon as the web UI writes to progress.txt, it gives the file ownership of “nobody:nobody” and permsissions of 0644.

The CLI then throws the following error:

Warningfopen(/home/user/public_html/magmigit/state/progress.txt)failed to open streamPermission denied in /home/user/public_html/magmigit/inc/magmi_loggers.php on line 13

Fatal error
Uncaught exception 'Exception' with message 'CANNOT WRITE PROGRESS FILE ' in /home/user/public_html/magmigit/inc/magmi_loggers.php:16
Stack trace
:
#0 /home/user/public_html/magmigit/cli/magmi.cli.php(80): FileLogger->__construct()
#1 {main}
  
thrown in /home/user/public_html/magmigit/inc/magmi_loggers.php on line 16

Also, I noticed in one of your commit messages you mentioned that the CLI can take chained commands. Is there any documentation regarding this yet? smile

 
Magento Community Magento Community
Magento Community
Magento Community
Magento Community
Magento Community
Back to top