Posting in the Magento forums has been disabled pending the implementation of a new and improved forum solution which should better serve the community.

For new questions please post at magento.stackexchange.com, the community-run support site for the Magento community. We will be providing updates on the new forum solution soon. For questions or concerns please email community@magento.com.

Magento Forum

Importing products - using a cron job to call Magento Dataflow import
 
aarpol
Jr. Member
 
Total Posts:  13
Joined:  2010-02-17
 

This is an old post now but is anyone still using this script with success on 1.4.0?

I’ve made the fixes above and am running the script using wget from the Linux shell to test it. The log file contains the following but the prices/stock values aren’t updated in the admin.

2010-03-03T09:34:15+00:00 DEBUG (7): Import Started
2010
-03-03T09:34:18+00:00 DEBUG (7): Skip import rowis not valid value "" for field "type"
2010-03-03T09:34:39+00:00 DEBUG (7): 20 Completed!!
2010-03-03T09:35:03+00:00 DEBUG (7): 40 Completed!!
2010-03-03T09:35:23+00:00 DEBUG (7): 60 Completed!!
2010-03-03T09:35:51+00:00 DEBUG (7): 80 Completed!!
2010-03-03T09:36:18+00:00 DEBUG (7): 100 Completed!!

Crucially, I’m not specifying a “type” column in the import - this is for stock level / price updates only.

Any ideas?

Aaron

 
Magento Community Magento Community
Magento Community
Magento Community
 
mChelevich
Jr. Member
 
Avatar
Total Posts:  20
Joined:  2009-03-31
 

I get error

SQLSTATE[42S22]Column not found1054 Unknown column 'e.display_price_group_0' in 'field list'
in product convert adapter’s finish() method (it dispatches event ‘catalog_product_import_after’, which is catched by all re-indexers)
I have flat catalog disabled.
Magento 1.3.2.4

Additionally, I can’t edit a product after the cron - shows the same error.

Any thoughts?

 
Magento Community Magento Community
Magento Community
Magento Community
 
shmk
Sr. Member
 
Total Posts:  297
Joined:  2009-10-28
 

Anyone has tested succesfully a cron job solution for importing products on Magento 1.4.1.0 ?

 
Magento Community Magento Community
Magento Community
Magento Community
 
Darsh
Member
 
Total Posts:  71
Joined:  2008-06-21
 

it says succes in cron schedules table but values not updated where do i check the execution log ?

 
Magento Community Magento Community
Magento Community
Magento Community
 
jamieea
Jr. Member
 
Total Posts:  4
Joined:  2010-03-22
 

Hi all,

I wonder if someone could help me please.  I’m trying to automate the execution of advanced profiles using the CaffeineJunkies_CronProfiles module to import some product stock information.  Thanks for writing the module - it looks good.

On Magento ver. 1.3.2.4, I have created an advanced profile with the following profile actions XML:

<action type="dataflow/convert_adapter_io" method="load">
    <var 
name="type">file</var>
    <var 
name="path">var/import</var>
    <var 
name="filename"><![CDATA[import_product_stocks.csv]]></var>
    <var 
name="format"><![CDATA[csv]]></var>
</
action>

<
action type="dataflow/convert_parser_csv" method="parse">
    <var 
name="delimiter"><![CDATA[,]]></var>
    <var 
name="enclose"><![CDATA["]]></var>
    <var name="
fieldnames">true</var>
    <var name="
store"><![CDATA[0]]></var>
    <var name="
number_of_records">1</var>
    <var name="
decimal_separator"><![CDATA[.]]></var>
    <var name="
adapter">catalog/convert_adapter_product</var>
   <var name="
method">parse</var>
</action>
<action type="
catalog/convert_adapter_product" method="parse">
</action>
<action type="
catalog/convert_adapter_product" method="finish">
</action>

The profile is enabled as a cron with the cron job checkbox checked, and the cromn expression is set to

*/* * * *

If I run the profile manually in the Magento Admin area, all is well and the stock levels are updated.

I’ve set the Magento cron.php to run every minute at the moment for testing purposes.  I’d expect to see some scheduling in the cron_schedule table in the database, but I can’t see any references to the profile.

I then thought the Magento cron.php may not be executing, so I:

a) sent an e-mail when it fired
b) added a newsletter and scheduled a send

I get e-mails when it fires and I also received the newsletter so all is well there.

Finally, I’ve checked for MySQL errors but none are reported.

Could anyone recommend any other courses of action to diagnose the issue here please?

Many thanks!

 
Magento Community Magento Community
Magento Community
Magento Community
 
babybidou
Jr. Member
 
Avatar
Total Posts:  15
Joined:  2010-04-26
 

Just to help you for cron debug, check the cron_schedule table in your MySQL database : if your job does not appear in there, it will not execute.

 
Magento Community Magento Community
Magento Community
Magento Community
 
D. Hes
Member
 
Total Posts:  33
Joined:  2010-06-24
Netherlands
 

I have tried to run an import via a script file and it has worked (in version 1.4.1.1). But after 5 hours of import the server terminated the script and now running the profile from the script does not work anymore. The profile does run, and lines are being imported into the tables dataflow_batch and dataflow_batch_import but they stay there and do not get importe into the products table.

It does work when i run the same profile manually from the admin-panel.

SOLVED
Apparently ther was a problem with the file. I manually cut a big file into pieces with notepad. Eventhough i saved it as .csv in UTF-8 the file was not read correctly. I copy-pasted the lines into another file created with pentaho Data Integration, then it did work.

 
Magento Community Magento Community
Magento Community
Magento Community
 
kamark
Jr. Member
 
Total Posts:  23
Joined:  2010-02-09
 

"jamieea” I have the same problem. Dont know whats wrong. Manually working but from cron its not. Its running but product is not updating.

 
Magento Community Magento Community
Magento Community
Magento Community
 
kamark
Jr. Member
 
Total Posts:  23
Joined:  2010-02-09
 

Ok just realized you will have to have a column as store and value as your store value for me its default. But the problem is when supplier push the csv file to the server doesnt have a store column. Anyone have any idea how to get around it?

Thanks.

 
Magento Community Magento Community
Magento Community
Magento Community
 
D. Hes
Member
 
Total Posts:  33
Joined:  2010-06-24
Netherlands
 
kamark - 15 September 2010 02:38 AM

Ok just realized you will have to have a column as store and value as your store value for me its default. But the problem is when supplier push the csv file to the server doesnt have a store column. Anyone have any idea how to get around it?

Thanks.

You could try to use a tool like Pentaho Data Integration or MS Excel to add the needed mandatory fields.

 
Magento Community Magento Community
Magento Community
Magento Community
 
kamark
Jr. Member
 
Total Posts:  23
Joined:  2010-02-09
 
D. Hes - 15 September 2010 02:44 AM

kamark - 15 September 2010 02:38 AM
Ok just realized you will have to have a column as store and value as your store value for me its default. But the problem is when supplier push the csv file to the server doesnt have a store column. Anyone have any idea how to get around it?

Thanks.

You could try to use a tool like Pentaho Data Integration or MS Excel to add the needed mandatory fields.

If i need to touch the file/manually edit the file why do i need to use cron?

 
Magento Community Magento Community
Magento Community
Magento Community
 
D. Hes
Member
 
Total Posts:  33
Joined:  2010-06-24
Netherlands
 

I suppose magento needs more info about a product to import it in its database, store is one of them. Since your supplier does not know that and probably will not add that field to it’s cvs. You’ll probably need to add such fields to taht csv to succesfully import the products.
Cron can be used to periodically import the products, so i guess you’ll allso need to change the csv periodically.

 
Magento Community Magento Community
Magento Community
Magento Community
 
kamark
Jr. Member
 
Total Posts:  23
Joined:  2010-02-09
 
D. Hes - 15 September 2010 03:18 AM

I suppose magento needs more info about a product to import it in its database, store is one of them. Since your supplier does not know that and probably will not add that field to it’s cvs. You’ll probably need to add such fields to taht csv to succesfully import the products.
Cron can be used to periodically import the products, so i guess you’ll allso need to change the csv periodically.

I can create a custom column from supplier which will be called store but will not have any value in it. So i am thinking to usw awk or shell script just add store value onto csv file but got no idea yet.

So it will be

1. Supplier pushing at 0400 hrs csv every day
2. set up cron for shell script editing the file 0430 hrs every day
3. set up cron for magento to update price and quantity at 0500 hrs

magento is very good and has very complex features but lacks some very simple features.

 
Magento Community Magento Community
Magento Community
Magento Community
 
jaggsr
Member
 
Total Posts:  39
Joined:  2009-03-25
Ahmedabad, India
 

I found this one: http://www.premasolutions.com/content/magento-dataflow-exportimport-form-command-line

Its very easy. One can run it from command line or can set to run via Cron job.

 
Magento Community Magento Community
Magento Community
Magento Community
 
OfficeSpot
Jr. Member
 
Avatar
Total Posts:  10
Joined:  2009-08-11
Dublin, Ireland
 

Hi there,

I followed and tested your script in in magento 1.4.1 however the products
don’t import and the import.log file show 403 permision dined errors.

Any idea what might be causing this.

This is an excellent feature and would really like to get in working.
------------------------------------------------------------------------------------------------------

Log File Output
------------------------------------------------------------------------------------------------------
2010-10-07T14:35:48+00:00 DEBUG (7):

2010-10-07T14:35:48+00:00 DEBUG (7): 7.55 mb - STARTING IMPORT
2010-10-07T14:35:48+00:00 DEBUG (7): 9.03 mb - Table dataflow_batch_import cleaned
2010-10-07T14:35:48+00:00 DEBUG (7): 9.03 mb - Preparing profile…
2010-10-07T14:36:18+00:00 DEBUG (7): 12 mb - ...Done
2010-10-07T14:36:18+00:00 DEBUG (7): 12 mb - Loaded batch id 208
2010-10-07T14:36:18+00:00 DEBUG (7): 17.12 mb - 0/20428
2010-10-07T14:36:18+00:00 DEBUG (7): 17.13 mb - 50/20428 [<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE html PUBLIC “-//W3C//DTD XHTML 1.0 Strict//EN”
“http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
<title>Access forbidden!</title>
<link rev="made" href="mailto:webmaster@localhost" />
<style type="text/css"><!--/*--><![CDATA[/*><!--*/
body { color: #000000; background-color: #FFFFFF; }
a:link { color: #0000CC; }
p, address {margin-left: 3em;}
span {font-size: smaller;}
/*]]>*/--></style>
</head>

<body>
<h1>Access forbidden!</h1>
<p>

You don’t have permission to access the requested directory.
There is either no index document or the directory is read-protected.

</p>
<p>
If you think this is a server error, please contact
the <a href="mailto:webmaster@localhost">webmaster</a>.

</p>

<h2>Error 403</h2>
<address>
<a href="/">dev.mydomain.com</a><br />

<span>Thu Oct 7 15:36:18 2010<br />

....................................................................................................................and..................................................................................

2010-10-07T14:43:18+00:00 DEBUG (7): 17.12 mb - Starting Mage_Dataflow_Model_Convert_Adapter_Io :: load
2010-10-07T14:43:18+00:00 DEBUG (7): 17.12 mb - Loaded successfully: “/var/www/mydomain/public/httpdocs/var/import/my_products.csv”.
2010-10-07T14:43:18+00:00 DEBUG (7): 17.12 mb - Starting Mage_Dataflow_Model_Convert_Parser_Csv :: parse
2010-10-07T14:43:18+00:00 DEBUG (7): 17.12 mb - Found 20428 rows.
2010-10-07T14:43:18+00:00 DEBUG (7): 17.12 mb - Starting catalog/convert_adapter_product :: parse
2010-10-07T14:43:18+00:00 DEBUG (7): 17.12 mb - Completed!
2010-10-07T15:04:03+00:00 DEBUG (7):

 
Magento Community Magento Community
Magento Community
Magento Community
Magento Community
Magento Community
Back to top