Posting in the Magento forums has been disabled pending the implementation of a new and improved forum solution which should better serve the community.

For new questions please post at magento.stackexchange.com, the community-run support site for the Magento community. We will be providing updates on the new forum solution soon. For questions or concerns please email community@magento.com.

Magento Forum

Page 2 of 11
Importing products - using a cron job to call Magento Dataflow import
 
pumba
Jr. Member
 
Total Posts:  28
Joined:  2008-05-14
 

Hi

Cool stuff, exactly what I need. Unfortunately it does not work yet for me.

After some debugging it seems like $batchModel->getAdapter() is always FALSE… Am I missing something in setup or it was changed in recent version of Magento? ( I use 1.1.3)

$batchModel Mage::getSingleton('dataflow/batch');
  if (
$batchModel->getId()) {
    
if ($batchModel->getAdapter()) {
// Never happens !!!!!
      
$batchId $batchModel->getId(); 
      
$batchImportModel $batchModel->getBatchImportModel();
      
$importIds $batchImportModel->getIdCollection();  

      
$batchModel Mage::getModel('dataflow/batch')->load($batchId);      
      
$adapter Mage::getModel($batchModel->getAdapter());
      foreach (
$importIds as $importId{
        $recordCount
++;
        
try{
          $batchImportModel
->load($importId);
          if (!
$batchImportModel->getId()) {
             $errors[] 
Mage::helper('dataflow')->__('Skip undefined row');
             continue;
          
}
 
Magento Community Magento Community
Magento Community
Magento Community
 
michaelc
Member
 
Avatar
Total Posts:  64
Joined:  2008-05-21
Qld, Australia
 

I have tested the .php script in a URL with the following result:
Warning: Unexpected character in input: ‘\’ (ASCII=92) state=1 in /home/musicsho/public_html/Cron_Import.php on line 7
Parse error: syntax error, unexpected T_IF in /home/musicsho/public_html/Cron_Import.php on line 8

Also tested via a cron job with the following error:
/bin/sh: Cron_Import.php: command not found
(it is finding the .php but the command isn’t running)

I am using Magento 1.1.2

Any ideas???

 
Magento Community Magento Community
Magento Community
Magento Community
 
michaelc
Member
 
Avatar
Total Posts:  64
Joined:  2008-05-21
Qld, Australia
 

OK, so I sorted out my errors with the cron process.
Now the CRON is running fine and there are no syntax errors.

The confirmation email is now stating:

--01:33:01-- https://www.my-domain.com.au/shop/cron_Import.php/?files=import_TEST_products.csv
=> `-’
Resolving www.musicshoponline.com.au… xx.xxx.xx.xxx
Connecting to www.musicshoponline.com.au|xx.xx.xx.xx|:xxx… connected.
HTTP request sent, awaiting response… 200 OK
Length: unspecified [text/html]
You are not a cron job!
0K 10.97 MB/s

01:33:01 (10.97 MB/s) - `-’ saved [23]

Any Ideas?

 
Magento Community Magento Community
Magento Community
Magento Community
 
michaelc
Member
 
Avatar
Total Posts:  64
Joined:  2008-05-21
Qld, Australia
 

OK, so I sorted out my IP error *slaps forehead*

...@srinigenie, thanks so much for the code. Works really well, exactly what I was looking for,

Cheers

 
Magento Community Magento Community
Magento Community
Magento Community
 
zadpro
Sr. Member
 
Avatar
Total Posts:  247
Joined:  2007-12-10
FL, USA
 

I have a weired problem that is driving me crazy.
I use the code posted by srinigenie. When I have my csv file with couple of items, it works perfectly fine. But when I try to use the complete csv file that has all the products (>9700) I get the error message 404 - File not found. I thought may be the file size is too big, even though I can run this big file directly from Magento admin without any problem, so I cut the number of products to half and continued......
Now here is what I have, if my csv file has 79 products, it works fine, if I have 80 products then I get the 404 error message again.
I checked the products on line 78-79-80, all three products have the same setup and there is no different in the csv file as far as special characters or so on.
Is this happening to me only?
Any idea what could be the reason?

Thanks

 
Magento Community Magento Community
Magento Community
Magento Community
 
joyously
Guru
 
Total Posts:  447
Joined:  2008-08-21
 
pumba - 27 August 2008 11:24 AM

After some debugging it seems like $batchModel->getAdapter() is always FALSE… Am I missing something in setup or it was changed in recent version of Magento? ( I use 1.1.3)

I have the same problem and I’m on 1.1.3 also.  I didn’t see anything in the core code that would indicate why the Adapter isn’t returned, but then I’m not real clear on what’s going on in the Mage code.
UPDATE:
I’ve been looking through the code and still don’t see why the Adapter is not returned.
But, I did figure out that the example given here does not incorporate the filename specified on the call into the profile. Whatever file is in the profile is the one that is read, not the one passed in the query string.
I really want to get this working, so I can import configurable products and set them up right on the import.

 
Magento Community Magento Community
Magento Community
Magento Community
 
VijayGadhavi
Jr. Member
 
Avatar
Total Posts:  11
Joined:  2008-07-17
India
 

Hey thank you for your help.

hey its very urgent please guide me. i want to set cron job file for the product import.

I have put Cron_Import.php file in my root folder as ur mention code. my improt file name is abc.xml
i have put that abc.xml file in var/import folder. and changed following line

$filename = Mage::app()->getRequest()->getParam(’files’);
chnaged by
$filename = Mage::app()->getRequest()->getParam(’abc.xml’);

now i m simply runing this file
http://<www.example.com>/Cron_Import.php/

but its give me message that “No file has been set!”. so anybody has any idea. please guide.

i dont know why that $filename shows blank. and i m not getting the “$profileId = 3;” what is that profile id?

 
Magento Community Magento Community
Magento Community
Magento Community
 
pezi5499
Jr. Member
 
Total Posts:  6
Joined:  2008-03-05
Munich
 

I found a solution for setting visibility and status in this cronjob product import setting with other languages:

Flashnet - 28 July 2008 12:56 AM

I used this cron job for importing products and have same issue, that visibility status is inactive. Did you find alredy a solution for this?
I use german version, but it seams, that tis not matters.

Look at /app/code/core/Mage/Catalog/Model/Product.Visibility.php and you will find the lines:

self::VISIBILITY_NOT_VISIBLE=> Mage::helper('catalog')->__('Nowhere'),
self::VISIBILITY_IN_CATALOG => Mage::helper('catalog')->__('Catalog'),
self::VISIBILITY_IN_SEARCH  => Mage::helper('catalog')->__('Search'),
self::VISIBILITY_BOTH       => Mage::helper('catalog')->__('Catalog, Search')

And in /app/code/core/Mage/Catalog/Model/Product.php the respective lines

self::STATUS_ENABLED    => Mage::helper('catalog')->__('Enabled'),
self::STATUS_DISABLED   => Mage::helper('catalog')->__('Disabled')

Now, in your csv-File, you have to have those english Strings instead of the German (or other languages) ones, so you have to replace them, either manually or with search and replace or by a script. For Example put ‘Catalog, Search’ instead of ‘Katalog, Suche’.
I am sure there is another easy solution for this, somehow activating the Magento-locale-Framework in the Cron_Import.php or so, but I haven’t found out yet how. Has anybody else?

Thanks anyway to all the help of you guys pointing us in the right direction…

 
Magento Community Magento Community
Magento Community
Magento Community
 
pezi5499
Jr. Member
 
Total Posts:  6
Joined:  2008-03-05
Munich
 
Vijay.Magento Master - 16 September 2008 03:21 AM


$filename = Mage::app()->getRequest()->getParam(’files’);
chnaged by
$filename = Mage::app()->getRequest()->getParam(’abc.xml’);

now i m simply runing this file
http://<www.example.com>/Cron_Import.php/

but its give me message that “No file has been set!”. so anybody has any idea. please guide.

i dont know why that $filename shows blank. and i m not getting the “$profileId = 3;” what is that profile id?

try

$filename 'abc.xml' ;
instead…
Profile Id is the number of your profile in the Magento Admin under Import / Export
 
Magento Community Magento Community
Magento Community
Magento Community
 
schnibble
Jr. Member
 
Total Posts:  2
Joined:  2008-08-20
 

I am also having problems with the script. My problem is that it skips all fields. Here is the log file:

2008-09-25T22:39:02+00:00 DEBUG (7): Import Started
2008
-09-25T22:39:03+00:00 DEBUG (7): Skip import rowrequired field "store" not defined
2008
-09-25T22:39:03+00:00 DEBUG (7): Skip import rowrequired field "store" not defined
2008
-09-25T22:39:03+00:00 DEBUG (7): Skip import rowrequired field "store" not defined
2008
-09-25T22:39:03+00:00 DEBUG (7): Skip import rowrequired field "store" not defined
2008
-09-25T22:39:03+00:00 DEBUG (7): Skip import rowrequired field "store" not defined
2008
-09-25T22:39:03+00:00 DEBUG (7): Starting Mage_Dataflow_Model_Convert_Parser_Csv :: parse
2008
-09-25T22:39:03+00:00 DEBUG (7): Found 5 rows
2008
-09-25T22:39:03+00:00 DEBUG (7): Starting catalog/convert_adapter_product :: parse
2008
-09-25T22:39:03+00:00 DEBUG (7): Import Completed

When I run same file through admin interface it works fine. Any ideas?

 
Magento Community Magento Community
Magento Community
Magento Community
 
schnibble
Jr. Member
 
Total Posts:  2
Joined:  2008-08-20
 

OK, I figured it out. It works when I try to import file that I exported from site. So it probably has to have exactly the same format as export. Hope it helps someone.

 
Magento Community Magento Community
Magento Community
Magento Community
 
Ben at Keeward
Jr. Member
 
Avatar
Total Posts:  18
Joined:  2008-06-26
Paris, France
 

Do you run this script from http or from a shell ?
I noticed also that $batchModel->getAdapter() is false when running the php script directly from a command line (but it does work when being ran from a wget).

It seems to me that invoking the script directly from the shell would be much secure and prevent it from being ran by potential hackers.
So I moved the php file out of Magento directory, used a chdir() to make it go back to the magento directory and replace this line :

$filename Mage::app()->getRequest()->getParam('files'); // set the filename that is to be imported - file needs to be present in var/import directory
by this one :
$filename $argv[1];
...without any success.
:-(
Anyone got an idea ?
Ben.
joyously - 05 September 2008 08:48 PM

pumba - 27 August 2008 11:24 AM

After some debugging it seems like $batchModel->getAdapter() is always FALSE… Am I missing something in setup or it was changed in recent version of Magento? ( I use 1.1.3)

I have the same problem and I’m on 1.1.3 also.  I didn’t see anything in the core code that would indicate why the Adapter isn’t returned, but then I’m not real clear on what’s going on in the Mage code.
UPDATE:
I’ve been looking through the code and still don’t see why the Adapter is not returned.
But, I did figure out that the example given here does not incorporate the filename specified on the call into the profile. Whatever file is in the profile is the one that is read, not the one passed in the query string.
I really want to get this working, so I can import configurable products and set them up right on the import.

 
Magento Community Magento Community
Magento Community
Magento Community
 
Ben at Keeward
Jr. Member
 
Avatar
Total Posts:  18
Joined:  2008-06-26
Paris, France
 

I think there is a nasty side effect here.

This line is not enough to set the filename :

$filename Mage::app()->getRequest()->getParam('files'); // set the filename that is to be imported - file needs to be present in var/import directory
somewhere (I have not found where yet) the program gets the filename from the query_string field ‘files’ and not from this variable only ($filename).

To check this, try this :
change the URL to :
cron_import.php?my_query_string_field_name=import.csv
and change the code to :

$filename Mage::app()->getRequest()->getParam('my_query_string_field_name'); // set the filename that is to be imported - file needs to be present in var/import directory
and you get a NULL getAdapter() : the program NEEDS a ’files‘ query_string field.

That also explains whey it does not work when ran from a command line…

Ben.

joyously - 05 September 2008 08:48 PM

pumba - 27 August 2008 11:24 AM

UPDATE:
I’ve been looking through the code and still don’t see why the Adapter is not returned.
But, I did figure out that the example given here does not incorporate the filename specified on the call into the profile. Whatever file is in the profile is the one that is read, not the one passed in the query string.
I really want to get this working, so I can import configurable products and set them up right on the import.
 
Magento Community Magento Community
Magento Community
Magento Community
 
Ben at Keeward
Jr. Member
 
Avatar
Total Posts:  18
Joined:  2008-06-26
Paris, France
 

If you really want to run your batch from a command line, you may try this (it works) :

if($argc<3)
{
    
die("[SYNTAX ERROR]\nSyntax: php import.php filename.csv profile_id");
}
chdir
('/my_path_to_magento_home_dir');

require_once 
'app/Mage.php';
umask(0);

Mage::app();

$filename $argv[1];
$profileId intval($argv[2]);
Mage::app()->getRequest()->setParam('files',$filename);

the key is this line :

Mage::app()->getRequest()->setParam('files',$filename);

...it is a workaround for the side effect I talked about earlier.

NB: I saved this php file outside Magento home directory so that I am sure it may not be accessed from the web.

Ben.

 
Magento Community Magento Community
Magento Community
Magento Community
 
Ben at Keeward
Jr. Member
 
Avatar
Total Posts:  18
Joined:  2008-06-26
Paris, France
 

Eventually it looks more like a ”hidden feature” than a side effect.
Look in /app/code/core/Mage/Dataflow/Model/Convert/Parser/Csv.php (line 79) :

if (Mage::app()->getRequest()->getParam('files')) {
            $file 
Mage::app()->getConfig()->getTempVarDir().'/import/'
                
urldecode(Mage::app()->getRequest()->getParam('files'));
            
$this->_copy($file);
        
}

The only way I found to make it work is to set Mage::app()->getRequest()->getParam(’files’) using Mage::app()->getRequest()->setParam(’files’, $filename)

Ben.

Benjamin Bellamy - 30 October 2008 06:21 AM

I think there is a nasty side effect here.

 
Magento Community Magento Community
Magento Community
Magento Community
Magento Community
Magento Community
Back to top
Page 2 of 11