Posting in the Magento forums has been disabled pending the implementation of a new and improved forum solution which should better serve the community.

For new questions please post at magento.stackexchange.com, the community-run support site for the Magento community. We will be providing updates on the new forum solution soon. For questions or concerns please email community@magento.com.

Magento Forum

Importing products - using a cron job to call Magento Dataflow import
 
lilmuckers
Jr. Member
 
Total Posts:  15
Joined:  2008-06-18
 

Hey guys, not sure if this is relevant for you, but it fixed my problem:

http://www.magentocommerce.com/boards/viewreply/125131/

 
Magento Community Magento Community
Magento Community
Magento Community
 
zadpro
Sr. Member
 
Avatar
Total Posts:  247
Joined:  2007-12-10
FL, USA
 
lilmuckers - 11 April 2009 03:08 AM

Hey guys, not sure if this is relevant for you, but it fixed my problem:


http://www.magentocommerce.com/boards/viewreply/125131/

It worked for me too. Thank you.

 
Magento Community Magento Community
Magento Community
Magento Community
 
LarMan128
Jr. Member
 
Total Posts:  14
Joined:  2008-09-14
 
barrmy - 03 November 2008 07:25 PM

*edit*

my error above was due to running the cron as a curl method, my host informed me that due to the size i had to use cli to avoid timeout

ben’s changes worked like a charm once i figured out the settings...49min for a full import of 3207 items

i have created the PHP file with the code posted on page 1. This works great if i type the address in my browser but I cant get it to work with cron...can someone help me with the correct command to set this up with cron. I got it to work with curl but like barrmy said it times out and doesnt finish. I only import about 80 out of about 1600. barrmy said they got it to work using cli...what would i do to set this up. any help would be appreciated.

Thanks

 
Magento Community Magento Community
Magento Community
Magento Community
 
LarMan128
Jr. Member
 
Total Posts:  14
Joined:  2008-09-14
 

nevermind...good it working! works great and is a huge time saver! thanks!

 
Magento Community Magento Community
Magento Community
Magento Community
 
eaugustin
Jr. Member
 
Total Posts:  9
Joined:  2008-11-06
 

Hi for whom it concerns.

It looks like this import script does not use the decimal point (,.) value for amounts from the importprofile.
I think you must always use decimal point!

Our decimal value was comma but it didn’t accept this!

I hope this is usefull for someone.

 
Magento Community Magento Community
Magento Community
Magento Community
 
brianf
Jr. Member
 
Total Posts:  4
Joined:  2009-03-15
 

This works great, except when I enable Flat Catalog Category.  Running the profile through the magento admin works either way. Does anyone have any ideas to fix this?

PHP Fatal error:  Call to undefined method Mage_Catalog_Model_Resource_Eav_Mysql4_Category_Flat::getResource() in /usr/local/apache/htdocs/app/code/core/Mage/Catalog/Model/Resource/Eav/Mysql4/Category/Flat.php on line 936
PHP Stack trace
:
PHP   1. {main}() /usr/local/apache/htdocs/DF_run.php:0
PHP   2. Mage_Catalog_Model_Convert_Adapter_Product
->saveRow($importData = *uninitialized*) /usr/local/apache/htdocs/DF_run.php:71
PHP   3. Mage_Core_Model_Abstract
->save() /usr/local/apache/htdocs/app/code/core/Mage/Catalog/Model/Convert/Adapter/Product.php:662
PHP   4. Mage_Eav_Model_Entity_Abstract
->save($object = *uninitialized*) /usr/local/apache/htdocs/app/code/core/Mage/Core/Model/Abstract.php:251
PHP   5. Mage_Catalog_Model_Resource_Eav_Mysql4_Product
->_beforeSave($object = *uninitialized*) /usr/local/apache/htdocs/app/code/core/Mage/Eav/Model/Entity/Abstract.php:891
PHP   6. Mage_Catalog_Model_Category
->verifyIds($ids = *uninitialized*) /usr/local/apache/htdocs/app/code/core/Mage/Catalog/Model/Resource/Eav/Mysql4/Product.php:119
PHP   7. Mage_Catalog_Model_Resource_Eav_Mysql4_Category_Flat
->verifyIds($ids = *uninitialized*) /usr/local/apache/htdocs/app/code/core/Mage/Catalog/Model/Category.php:609
 
Magento Community Magento Community
Magento Community
Magento Community
 
amartinez
Guru
 
Avatar
Total Posts:  347
Joined:  2008-11-14
Spain
 

Can you test this version of import scripts?

http://www.magentocommerce.com/boards/viewreply/131686/

Thank you!

 
Magento Community Magento Community
Magento Community
Magento Community
 
cheeseandrice
Sr. Member
 
Avatar
Total Posts:  120
Joined:  2009-01-05
 

Can someone share what they did to get this running on cron. I want to start EXPORTING inventory then sending the file by FTP to my client to upload to their ERP. So a CronExport.php. Please advice. I would love to see your files if you would be so kind.

 
Magento Community Magento Community
Magento Community
Magento Community
 
BetaJ
Jr. Member
 
Avatar
Total Posts:  7
Joined:  2007-10-08
Hamburg, Germany
 

Hello,
may somebody find this useful.
I get the script working by changing it that way:

importperskript.php

<?php

  ini_set
('memory_limit''3G');

  require_once 
'app/Mage.php';
  
umask(0);

  
//$_SERVER['SERVER_PORT']='443';
  
Mage::app()->setCurrentStore(Mage_Core_Model_App::ADMIN_STORE_ID);

  
$profileId 3//put your profile id here
 
  
$profile Mage::getModel('dataflow/profile');
  
  
$userModel Mage::getModel('admin/user');
  
$userModel->setUserId(0);
  
Mage::getSingleton('admin/session')->setUser($userModel);
  
  if (
$profileId{
    $profile
->load($profileId);
    if (!
$profile->getId()) {
      Mage
::getSingleton('adminhtml/session')->addError('The profile you are trying to save no longer exists');
    
}
  }

  Mage
::register('current_convert_profile'$profile);

  
$profile->run();
  
  
$batchModel Mage::getSingleton('dataflow/batch');
  if (
$batchModel->getId()) {
    
if ($batchModel->getAdapter()) {
      $batchId 
$batchModel->getId(); 
      
$batchImportModel $batchModel->getBatchImportModel();
      
$importIds $batchImportModel->getIdCollection();  

      
$batchModel Mage::getModel('dataflow/batch')->load($batchId);      
      
$adapter Mage::getModel($batchModel->getAdapter());
      foreach (
$importIds as $importId{
        $recordCount
++;
        
try{
          $batchImportModel
->load($importId);
          if (!
$batchImportModel->getId()) {
             $errors[] 
Mage::helper('dataflow')->__('Skip undefined row');
             continue;
          
}

          $importData 
$batchImportModel->getBatchData();
          try 
{
            $adapter
->saveRow($importData);
          
catch (Exception $e{
            
echo $e->getMessage();
            continue;
          
}
        
          
if ($recordCount%20 == 0{
            
echo 'Processed: '.$recordCount ''.chr(13);
          
}
        } 
catch(Exception $ex{
          
echo 'Record# ' $recordCount ' - SKU = ' $importData['sku']' - Error - ' $ex->getMessage();   
        
}
      }
      
foreach ($profile->getExceptions() as $e{
        
echo $e->getMessage();
      
}
      
    }
  }
  
echo 'Import Completed';
The profile 3 is set to import a fix local file, not interactive.

And then run it from the command line using nohup, so I can close the ssh-session.

cd [your magento server home]
nohup php importperskript
.php >/var/import.log &

you can monitor /var/import.log for processing.

The line

ini_set('memory_limit''3G');
is neccessary to provide as much memory as possible.

It works, now it comes to nearly 100.000 products. But I want 297.000 products.
I am new at magento, does anyone know, whether such a large number of products
is possible?

And may somebody can say how to reduce the memory used by this script.
I think about setting a memory consuming variable to null and or use unset().

Any Idea?

Greetings

BetaJ

 
Magento Community Magento Community
Magento Community
Magento Community
 
xer0x
Jr. Member
 
Total Posts:  9
Joined:  2009-05-19
 

Thank-you! This is so helpful!

 
Magento Community Magento Community
Magento Community
Magento Community
 
Cristofferson
Jr. Member
 
Total Posts:  5
Joined:  2008-10-25
 

Hi! Thanks for this great contribution!

I have a weird warning when i try to import products via cronjob: 

-Powered-ByPHP/5.2.8Content-typetext/html <br /><b>Fatal error</b>: Uncaught exception 'Exception' with message 'Warning: strpos() [<a href='function.strpos'>function.strpos</a>]: Empty delimiter in /home/apostol/public_html/lib/Zend/Controller/Request/Http.php on line 487' in /home/apostol/public_html/app/code/core/Mage/Core/functions.php:239Stack trace:

Executing the profile manually always worked.

Help me please.

 
Magento Community Magento Community
Magento Community
Magento Community
 
uke_boy
Jr. Member
 
Total Posts:  26
Joined:  2009-02-20
Kiev, Ukraine
 

Hello,
may somebody find this useful.
I get the script working by changing it that way:

importperskript.php

Hi, guys
Does anybody knows why it may happens: I changed admin ID and script ID and then tried to run this script, but here:

$profile->run();
  
  
$batchModel Mage::getSingleton('dataflow/batch');
  if (
$batchModel->getId()) {
    
if ($batchModel->getAdapter()) {
there is nothing going on: $batchModel->getAdapter() always False.
I’m using 1.3.2.2

EDIT: Issue is SOLVED (just forgot to save profile after changing interactive mode to predefined file name)

 
Magento Community Magento Community
Magento Community
Magento Community
 
pranav
Jr. Member
 
Total Posts:  1
Joined:  2009-08-16
 

Hi All,

Have any one of you have found any solution about this issue? Please provide any suggestion/hint. Any help will be appreciated.

Thanking you and waiting for quick reply,

Warm regards,
PranaV

 
Magento Community Magento Community
Magento Community
Magento Community
 
Draekonyss
Jr. Member
 
Total Posts:  1
Joined:  2009-06-26
 

Here is the code to implement in a proper way a Schedule Cron Task that run ‘import profile’

1st, create a module
For example, name it ‘Mycron’
put it in your app/code/local/Mynamespace folder
You must have something like this

/app/code/local/Mynamespace
-----etc
-----Model

In your config.xml, put this

<config>
    <
modules>
        <
Mynamespace_Mycron>
            <
version>0.0.1</version>
        </
MyNameSpace_Mycron>
    </
modules>
    <global>
        <
models>
            <
mynamespacemycron>
                <class>
Mynamespace_Mycron_Model</class>
            </
mynamespacemycron>
        </
models>
    </global>
   <
crontab>
        <
jobs>
            <
mycron_import_product>
                <
schedule><cron_expr>* * * * *</cron_expr></schedule>
                <
run><model>mynamespacemycron/cron::myMethod</model></run>
            </
mycron_import_product>
        </
jobs>
    </
crontab>
</
config>

then create your Model ‘Cron’ in your Model folder
call it Cron.php

class Mynamespace_Mycron_Model_Cron
{
         
public function myMethod()
        
{
                $profile 
Mage::getModel('dataflow/profile');
                
$userModel Mage::getModel('admin/user');
                
$userModel->setUserId(0);
                 
Mage::getSingleton('admin/session')->setUser($userModel);

                 
$profileId 9//put here the profile_id to run
                 
if ($profileId)
                 
{
                       $profile
->load($profileId);
                       if (!
$profile->getId())
                       
{
                              Mage
::getSingleton('adminhtml/session')->addError('The profile you are trying to save no longer exists');
                       
}
                 }
                 
//Mage::register('current_convert_profile', $profile); not necessary, seems to give me some errors
                 
$profile->run();
         
}
}

And that’s all. You don’t need to edit the main cron.php, and your code is portable !

 
Magento Community Magento Community
Magento Community
Magento Community
 
javiforos
Jr. Member
 
Total Posts:  5
Joined:  2009-08-21
 

Tanks BetaJ is very useful, one question,

In my script it does not get some attributes because of the value of options in Spanish (example: status:habilitado), it does not recognize that value for the option and set it as null)

In native process it sets some properties before calling to saveRow, can anyone help to generate same settings?

thanks in advance

BetaJ - 16 July 2009 04:01 AM

Hello,
may somebody find this useful.
I get the script working by changing it that way:

.........

BetaJ

 
Magento Community Magento Community
Magento Community
Magento Community
Magento Community
Magento Community
Back to top