Magento - Import Speed / Performance optimization - Magento DataFlow - eCommerce Software for Growth

Support
Search

Search the Magento Web site
Log In

Username or Email

Password:

Forgot Password

or
Register

Skip to site navigation »
Skip to main content »
Skip to footer content »

Magento Forum

Our new hosted solution for small & emerging businesses

Poll
Do you think the product import in Magento is too slow?
Yes, it should be improved!	580
No, it’s fine. There are more important things to work on.	18
What are you talking about?	1
Total Votes: 599	You must be a logged-in member to vote

Import Speed / Performance optimization

Discovery Total Posts: 767 Joined: 2007-12-29 Ignore user	I tried running that query - someone knows their sql, but not me! I had zero rows affected no matter how I tried to run it, and I did setup the temp table and import into that so I am happy with my syntax. I ended up using a few simple queries, one to get the product_id from the sku, then updated everything after that, less elegantly than the one statement above. I would still like to see what you got though, what works for you. Someone - hint, hint, needs to put a wiki article on this as everyone is left to find out for themselves or use the ‘dataflow’ route. Signature How to ask questions
	Posted: June 11 2009 \| top \| # 16

michael_g08 Total Posts: 108 Joined: 2009-03-06 Ukraine Ignore user	Hi. That’s the simple script I’ve made for one of community members. May be it would be useful. Regards, Michael. File Attachments script.zip (File Size: 2KB - Downloads: 1720)
	Posted: June 12 2009 \| top \| # 17

Discovery Total Posts: 767 Joined: 2007-12-29 Ignore user	Very neat! Impressive, clean code - if only that had been up here earlier. I really would advise posting that in the ‘wiki’ for the next person. Thanks muchly. Signature How to ask questions
	Posted: June 12 2009 \| top \| # 18

michael_g08 Total Posts: 108 Joined: 2009-03-06 Ukraine Ignore user	I’m glad to help. May be You are right about Wiki. But I have not much time for now. Pity :-( I’ll try to do it as soon as I can. Regards, Michael.
	Posted: June 12 2009 \| top \| # 19

Discovery Total Posts: 767 Joined: 2007-12-29 Ignore user	Done it! http://www.magentocommerce.com/wiki/stock_level_import_script Signature How to ask questions
	Posted: June 12 2009 \| top \| # 20

michael_g08 Total Posts: 108 Joined: 2009-03-06 Ukraine Ignore user	@Discovery I thank You for that. And I’m pleased You mentioned my name there. That’s great and I hope it will help… Regards, Michael.
	Posted: June 12 2009 \| top \| # 21

michael_g08 Total Posts: 108 Joined: 2009-03-06 Ukraine Ignore user	@Discovery Hi. Here is an updated full version of script. Now it can do: - Update products in magento database - Insert products if they not exist in magento DB - Insert/update MANUFACTURER field - Update QTY, PRICE, DESCRIPTION, SHORT_DESCRIPTION, and some my custom attributes. Update part is direct-SQL. Insert part uses magento API it’s based on code written by blakew. Hope it will helps somebody. Regads, Michael. File Attachments script.zip (File Size: 14KB - Downloads: 1369)
	Posted: June 15 2009 \| top \| # 22

amsoftuk Total Posts: 2 Joined: 2009-03-30 Ignore user	Hi, a couple of small refinements I have added to this script, that I don’t know if you would find useful. I have added them inline here rather than added them to the script, as I am using the earlier version of the script. (which does everything I need). First, at the top of the script include_once 'path/to/Mage.php'; umask(0); Mage::app('default'); $config = Mage::getConfig()->getResourceConnectionConfig('core_write'); $base = Mage::getBaseDir(); ////////////////GLOBAL DEFINITIONS///////////////////// //DATABASE SETTINGS $dbConfig = array( 'host' => $config->host, 'username' => $config->username, 'password' => $config->password, 'dbname' => $config->dbname, 'driver_options'=> array(PDO::MYSQL_ATTR_INIT_COMMAND => 'SET NAMES UTF8') ); This means you don’t need to hardcode the db info in the script. Needless to say, the include for Mage.php below the initial config needs to go. `function updateTempTableFromFile($db_magento, $path){ $db_magento->query("CREATE TEMPORARY TABLE IF NOT EXISTS temp_table (sku CHAR(50), qty INT(32))");` This creates the temp_table for you, if it isn’t there already, as a temporary table. Again fairly obviously you’ll need to change the columns to the ones you actually want to use. Hope that helps someone
	Posted: June 15 2009 \| top \| # 23

nafnaf1000 Total Posts: 204 Joined: 2008-02-21 Ignore user	Dummy requesting help. Sorry to bother you all. This is exactly what I need to do in order to get my site up and running. But I have no idea how to work this code. Can some one please help me to set this up. I need to upload new items once a day. and I need to upload price and qty every hour..... can anyone please help.
	Posted: June 17 2009 \| top \| # 24

boxer1732 Total Posts: 12 Joined: 2009-02-04 Ignore user	Michael - this code is just what we needed! Thanks for your time and effort.
	Posted: June 18 2009 \| top \| # 25

michael_g08 Total Posts: 108 Joined: 2009-03-06 Ukraine Ignore user	Glad to help, boxer !
	Posted: June 18 2009 \| top \| # 26

sphays Total Posts: 6 Joined: 2009-06-10 Ignore user	I have tried this script several times with different tweaks and parameters, but keep getting the same error: Fatal error: Uncaught exception ‘Zend_Db_Statement_Exception’ with message ‘SQLSTATE[28000]: Invalid authorization specification: 1045 Access denied for user ‘sstaley_test’@’webservices.servername.com’ (using password: YES)’ in /home/sstaley/public_html/test29464/lib/Zend/Db/Statement/Pdo.php:238 Stack trace: #0 /home/sstaley/public_html/test29464/lib/Zend/Db/Statement.php(283): Zend_Db_Statement_Pdo->_execute(Array) #1 /home/sstaley/public_html/test29464/lib/Zend/Db/Adapter/Abstract.php(457): Zend_Db_Statement->execute(Array) #2 /home/sstaley/public_html/test29464/lib/Zend/Db/Adapter/Pdo/Abstract.php(230): Zend_Db_Adapter_Abstract->query(’LOAD DATA INFIL...’, Array) #3 /home/sstaley/public_html/test01.php(62): Zend_Db_Adapter_Pdo_Abstract->query(’LOAD DATA INFIL...’) #4 /home/sstaley/public_html/test01.php(20): updateTempTableFromFile(Object(Zend_Db_Adapter_Pdo_Mysql), ‘/import/sam.csv’) #5 {main} thrown in /home/sstaley/public_html/test29464/lib/Zend/Db/Statement/Pdo.php on line 238 anyone else experience this or know how to fix. Thanks for any help
	Posted: June 29 2009 \| top \| # 27

Discovery Total Posts: 767 Joined: 2007-12-29 Ignore user	Yeah. You got your login wrong. Make sure your web services user/pass(apikey) is what it should be, give the role full permission for everything and all should be cushty. Signature How to ask questions
	Posted: June 29 2009 \| top \| # 28

michael_g08 Total Posts: 108 Joined: 2009-03-06 Ukraine Ignore user	Disovery is wright. sphays. You should contact Your web sevices provider. You are not only one who get this error. You just have not all the permissions. If You run LOAD DATA INFILE query in phpMyAdmin You will get the same error.
	Posted: June 30 2009 \| top \| # 29

sphays Total Posts: 6 Joined: 2009-06-10 Ignore user	as I stated in the post...we have verified the database connection credentials and the database user permissions serveral times over and continue to get this error.... I am not sure what else to try
	Posted: June 30 2009 \| top \| # 30

Magento Community

Magento Community

Back to top

Errors on Dashboard

Skip import row, is not valid value "" for field "type"

© Copyright Magento Inc.
Privacy Policy|Terms of Service Magento Community Count
|745 users currently online|519713 forum posts