Magento Forum

   
Page 1 of 2
which is the best way for fast importing products (more than 1k)
 
steppenwolf
Jr. Member
 
Total Posts:  16
Joined:  2010-02-17
 

Hi all , i’m a newbie magento php coding , so i’m looking a way for import bulk products from an xml data.
i wrote a code using simplxml and it works perfectly (easy configurable and fast) but when i connect to magento api via soap , it take about 1second per products importing to magento.
Even i dont parse an xml , a simple api connection take 10seconds.

$proxy = new SoapClient(’http://127.0.0.1/magento/api/v2_soap/?wsdl’);
$sessionId = $proxy->login(’user’, ‘pass’);
$customerinfo = $proxy->customerCustomerInfo($sessionId,1);
print_r($customerinfo);

are these durations normal? can i speed it up by changing mysql conf (i use with default wamp configurations) ,
which method should i use for increase speed?  i looked a way for direct sql coding but , magento uses a lot of table for one product (attributes, stocks, description, url-key etc....) an it seems too complex for this process.

Thanks for your replies…

 
Magento Community Magento Community
Magento Community
Magento Community
 
steppenwolf
Jr. Member
 
Total Posts:  16
Joined:  2010-02-17
 

is there anbody here…

 
Magento Community Magento Community
Magento Community
Magento Community
 
steppenwolf
Jr. Member
 
Total Posts:  16
Joined:  2010-02-17
 

i think direct sql is very very fast then soap or mage classes

--------------------------------------
<?php
$time1 = time();
require_once 'c:/wamp/www/magento/app/Mage.php';
Mage::app();
$cModel = Mage::getModel('catalog/product');

for ( $counter = 100; $counter <= 500; $counter += 1) {

$cModel->load($counter);
echo 'Product name: ' . $cModel->getName() . $cModel->getPrice().'</br>';
}
$time2 = time();
$duration = $time2 - $time1;
echo $duration;
?>
----------------------------------------- duration : 9

------------------------------------------
while ($db_field = mysql_fetch_assoc($result)) {
$SQL2 = “SELECT * FROM catalog_product_index_price where entity_id=”.$db_field[’entity_id’];
$result2 = mysql_query($SQL2);
$db_field2 = mysql_fetch_assoc($result2);

print $db_field[’entity_id’] . “<BR>”;
print $db_field[’sku’] . “<BR>”;
print $db_field2[’price’] . “<BR>”;

}
------------------------------------------duration:0 second

but how can i put all product attributes (sku , name , description, price , url_key, product_type....) at the same time via direct sql coding.

 
Magento Community Magento Community
Magento Community
Magento Community
 
uqppa
Jr. Member
 
Total Posts:  2
Joined:  2010-01-21
 

Why don’t you use System -> Import / Export -> Profiles -> Import All Products from Admin Panel ? Anyway, even doing it this way, Magento needs its time to build all the structure for the imported products. I don’t recommend you importing products directly from sql statements, since it may not be compatible with further version of Magento or even with installed extensions.

 
Magento Community Magento Community
Magento Community
Magento Community
 
steppenwolf
Jr. Member
 
Total Posts:  16
Joined:  2010-02-17
 

first of all , i tried this method you said , but it is to slow for importing 3000products

 
Magento Community Magento Community
Magento Community
Magento Community
 
jgarite
Jr. Member
 
Avatar
Total Posts:  23
Joined:  2009-12-11
 

I would be interested in knowing how you find an answer to this issue becasue I am looking into getting this software to replace our current software and we often import/update anywhere from 1000->10,000 items at once....

 
Magento Community Magento Community
Magento Community
Magento Community
 
jgarite
Jr. Member
 
Avatar
Total Posts:  23
Joined:  2009-12-11
 
uqppa - 16 March 2010 02:57 AM

Why don’t you use System -> Import / Export -> Profiles -> Import All Products from Admin Panel ? Anyway, even doing it this way, Magento needs its time to build all the structure for the imported products. I don’t recommend you importing products directly from sql statements, since it may not be compatible with further version of Magento or even with installed extensions.

I did not see this option in the demo for the community version...is this only in enterprise?

 
Magento Community Magento Community
Magento Community
Magento Community
 
chaddro
Jr. Member
 
Total Posts:  25
Joined:  2007-12-20
 

You can import our products through System -> Import/Export -> Profiles.

However, this is STILL very slow. About 1 per second even with the indexes “update on save” set to “manual”.

Anyone have a solutions yet?

 
Magento Community Magento Community
Magento Community
Magento Community
 
nwaelti
Jr. Member
 
Total Posts:  11
Joined:  2010-03-30
 

I’m also looking for a solution as I need to import 25K+ products… in the same time, I’d also like to improve the indexer…
Anyone has an idea?

 
Magento Community Magento Community
Magento Community
Magento Community
 
hi_meral
Jr. Member
 
Avatar
Total Posts:  13
Joined:  2010-04-28
 

http://www.magentocommerce.com/magento-connect/eCommerce+Software+Solutions+Online/extension/4087/product_importer_xml_csv

 
Magento Community Magento Community
Magento Community
Magento Community
 
16325860903
Sr. Member
 
Avatar
Total Posts:  163
Joined:  2011-02-02
Warsaw, Poland
 

Advanced Settings of GoMage Feed Pro perhaps could help you
Number of Products
This option allows you to optimize file creation for low memory servers or for stores with a large quantity of
products.
Set “Number of Products” equal 50 or 100 (or more) and generate the Feed file.
You will get the Feed file with all 10.000 products.
Export Disabled and Out of stock Products
Select “Yes” if you want to export Disabled and Out of stock products into Feed file.
You will be able to get more details on our page. You will find there live demo and users manual.

 
Magento Community Magento Community
Magento Community
Magento Community
 
16325860903
Sr. Member
 
Avatar
Total Posts:  163
Joined:  2011-02-02
Warsaw, Poland
 

Advanced Settings of GoMage Feed Pro perhaps could help you
Number of Products
This option allows you to optimize file creation for low memory servers or for stores with a large quantity of
products.
Set \"Number of Products\” equal 50 or 100 (or more) and generate the Feed file.
You will get the Feed file with all 10.000 products.
Export Disabled and Out of stock Products
Select \"Yes\" if you want to export Disabled and Out of stock products into Feed file.
You will be able to get more details on our page. You will find there live demo and users manual.

 
Magento Community Magento Community
Magento Community
Magento Community
 
dweeves
Enthusiast
 
Total Posts:  877
Joined:  2010-06-26
FRANCE
 

try Magmi !!! see this post
it’s free, it’s fast and has a lot of features smile

Magmi currently handles CSV but it’s based on an open plugin architecture.
I can provide some support for teaching you how to create a custom datasource based on your XML feed.

 
Magento Community Magento Community
Magento Community
Magento Community
 
ZenMasta
Sr. Member
 
Avatar
Total Posts:  108
Joined:  2010-03-12
 

[edit] I didn’t realize how old this thread was before posting :/

this is probably not much help for you. But just thought I’d share my workflow with you.

I get a pretty substantial xml file weekly. I have a java app that parses the xml and maps various fields (with some concatenation) to a table in a .mdb file. The mdb is used by our back office system. the xml has new products and updates to old (price changes, release dates etc). This java utility only takes a minute or two.

I take the .mdb and create a mapping for my dataflow template save that as a csv (value delimiter ^ and enclose values in ~).

My csv has about 4400 (includes some discontinued items). data flow import takes about 40 minutes. They key is setting the Number of records (under profile information) from 1 to something much greater. I do 500 at a time. If you leave it at 1 at a time, then expect dataflow to take about 4 hours on a list this size.

 
Magento Community Magento Community
Magento Community
Magento Community
 
dweeves
Enthusiast
 
Total Posts:  877
Joined:  2010-06-26
FRANCE
 

4400 items, magmi could do it in about 1mn or less depending on attribute number !!!
it has also a lot of neat features and an open plugin architecture.
really worth the try !!!

 
Magento Community Magento Community
Magento Community
Magento Community
 
Zolli
Jr. Member
 
Total Posts:  6
Joined:  2012-09-20
 

Basically you can speed up your import in different ways. You can use some tools or services or you can write a code. I prefer tools and services. As for me they are more reliable than codes. Personally I use this extension. the reason is that i wrote a code a few month ago and all my import failed. I had to wait until the import is over and i noticed that nothing had changed - all the products were not imported. I tried to do it one more time..but nothing happened. Since then i don’t use php codes.

 
Magento Community Magento Community
Magento Community
Magento Community
Magento Community
Magento Community
    Back to top
Page 1 of 2