Posting in the Magento forums has been disabled pending the implementation of a new and improved forum solution which should better serve the community.

For new questions please post at magento.stackexchange.com, the community-run support site for the Magento community. We will be providing updates on the new forum solution soon. For questions or concerns please email community@magento.com.

Magento Forum

Page 10 of 11
Importing products - using a cron job to call Magento Dataflow import
 
OfficeSpot
Jr. Member
 
Avatar
Total Posts:  10
Joined:  2009-08-11
Dublin, Ireland
 

I see this:

{"ajaxExpired”:1,"ajaxRedirect”:"http:\/\/dev.mydomain\/index.php\/admin\/index\/login\/key\/c4763f4bcfc014ed66ea1edfadb45d61\/"}

 
Magento Community Magento Community
Magento Community
Magento Community
 
OfficeSpot
Jr. Member
 
Avatar
Total Posts:  10
Joined:  2009-08-11
Dublin, Ireland
 

When I run mag_login.php with “php mag_login.php” it just runs no error on screen, no inputs to import.log file

If I run it from the browser I get the message: The page isn’t redirecting properly

Discussion continued on: http://www.h-o.nl/blog/automatic-import-with-magento-using-ssh-no-browser-needed/

 
Magento Community Magento Community
Magento Community
Magento Community
 
sucitivel
Jr. Member
 
Avatar
Total Posts:  15
Joined:  2009-07-21
Chicago, IL
 

This was a little too hacked together for me, and not really what i was looking for.  Unless you’re looking for the advanced features offered through the import profiles, you can get away with a simple import with a much simpler algorithm.  If your column headers match their respective attributes in the csv file, you can just do the following:

#!/usr/bin/php

<?php
function create_import_data($headers$row{
    $merged 
= array();
    
    foreach(
$headers as $k=>$header{
        $merged[trim
($header)$row[$k];
    
}
    
return $merged;
}
define
('IMPORT_FILE''/srv/httpd/example.com/magento/var/import/import-20101220182327-1_cardinal_binders_bz.2010.12.20.final.csv');

include(
'app/Mage.php');
ini_set('display_errors'1);
ini_set('error_reporting'7);

Mage::app()->setCurrentStore(Mage_Core_Model_App::ADMIN_STORE_ID);

$adapter Mage::getModel('catalog/convert_adapter_product');

$f fopen(IMPORT_FILE'r');
$headers fgetcsv($f);

while(
$data fgetcsv($f)) {
    $merged 
create_import_data($headers$data);
    
$adapter->saveRow($merged);
    echo 
"Saved sku $merged[sku]\r\n";
}

?>

This will take each row of data, merge it into a formatted associative array, then use the product adapter to save it… quick… dirty… done.

 
Magento Community Magento Community
Magento Community
Magento Community
 
Jose M Moreno
Member
 
Total Posts:  58
Joined:  2008-09-01
 

I use this last code, it\’s very usefull for me but if I print_r($merged) it\’s fine, all info of file it\’s shown but SaveRow truncate fields description, Any Idea?

Thanks and best regards,

while($data = fgetcsv($f,0,\’;\’,\’\"\’)) {

$merged = create_import_data($headers, $data);

$adapter->saveRow($merged);

print_r($merged);

 
Magento Community Magento Community
Magento Community
Magento Community
 
StevenThys
Jr. Member
 
Total Posts:  7
Joined:  2010-01-20
 

Hello,

Is there someone who could help me.

I need a script so i can import a csv or a xml-file for example each hour
When there is a sku in the file only the stock and the price is updated. All the other things stay the same in my shop.

On this way i can connect my POS with my website.

Is there someone who can provide me a doc/pdf with the steps how i can do this?

regards,

Steven

 
Magento Community Magento Community
Magento Community
Magento Community
 
anfrox
Jr. Member
 
Total Posts:  5
Joined:  2011-02-20
 

thanks for the great script! The only problem that i have is that the script doesn’t update any existing product values while not showing any errors in the log file, what can be the issue?

 
Magento Community Magento Community
Magento Community
Magento Community
 
amartinez
Guru
 
Avatar
Total Posts:  347
Joined:  2008-11-14
Spain
 

Hi, this is a old post, please revise these links:

http://www.magentocommerce.com/boards/viewthread/227005/

http://www.magentocommerce.com/boards/viewreply/332097/

http://www.magentocommerce.com/boards/viewreply/333582/

I hope it helps!

 
Magento Community Magento Community
Magento Community
Magento Community
 
dweeves
Enthusiast
 
Total Posts:  877
Joined:  2010-06-26
FRANCE
 

Magmi is designed to use magento dataflow format, performing much faster & having abiilty to adapt to any other csv format or able to get data from external database.
See forum entry (the most popular in dataflow thread) & wiki

Magmi is already used in production by magento partners companies, is free & reliable

 
Magento Community Magento Community
Magento Community
Magento Community
 
Berlic
Jr. Member
 
Total Posts:  1
Joined:  2011-12-25
 

Hello!
I try to automate magento dataflow profile. I use CronProfiles extension.
The action XML is:

<action type="dataflow/convert_adapter_io" method="load">
    <var 
name="type">file</var>
    <var 
name="path">var/import</var>
    <var 
name="filename"><![CDATA[import_product_stocks.csv]]></var>
    <var 
name="format"><![CDATA[csv]]></var>
</
action>

<
action type="dataflow/convert_parser_csv" method="parse">
    <var 
name="delimiter"><![CDATA[,]]></var>
    <var 
name="enclose"><![CDATA["]]></var>
    <var name="
fieldnames">true</var>
    <var name="
store"><![CDATA[0]]></var>
    <var name="
number_of_records">1</var>
    <var name="
decimal_separator"><![CDATA[.]]></var>
    <var name="
adapter">catalog/convert_adapter_product</var>
    <var name="
method">parse</var>
</action>

<action type="
catalog/convert_adapter_product" method="parse">
</action>

<action type="
catalog/convert_adapter_product" method="finish">
</action>

When I run profile using admin console—everything works as expected.
When I run profile via cron.php—it shows no error, but finishes half-done.
I have txt-data loaded into magento_root/vat/tmp/batch_NN.tmp and nothing else.

Any ideas why the actual dataflow is not done? Thanks in advance.

UPD: SOLVED!
The reason was that there was no “store” parameter in csv file.

 
Magento Community Magento Community
Magento Community
Magento Community
 
farhanhafeez
Jr. Member
 
Total Posts:  1
Joined:  2011-12-01
 

Referring to the code in post below, does this work with orphan rows? I am having issues. Basically, orphan rows does not get populated. Any idea what can be done?

srinigenie - 22 May 2008 10:53 AM

Pasting the code that helped me to import products using a cron-

STEP 1: Save the below file as Cron_Import.php on the Magento base folder.

<?php
//uncomment when moved to server - to ensure this page is not accessed from anywhere else
//if ($_SERVER['REMOTE_ADDR'] !== '<your server ip address') {
//  die("You are not a cron job!");
//}

require_once 'app/Mage.php';
// wget -O - http://<www.example.com>/Cron_Import.php/?files=3XSEEEE.csv
  
umask(0);

  
//$_SERVER['SERVER_PORT']='443';
  
Mage::app();

  
$profileId 3//put your profile id here
  
$filename Mage::app()->getRequest()->getParam('files'); // set the filename that is to be imported - file needs to be present in var/import directory  
  
if (!isset($filename))  {
 
die("No file has been set!");
  
}
  $logFileName
$filename.'.log';  
  
$recordCount 0;

  
Mage::log("Import Started",null,$logFileName);  
 
  
$profile Mage::getModel('dataflow/profile');
  
  
$userModel Mage::getModel('admin/user');
  
$userModel->setUserId(0);
  
Mage::getSingleton('admin/session')->setUser($userModel);
  
  if (
$profileId{
    $profile
->load($profileId);
    if (!
$profile->getId()) {
      Mage
::getSingleton('adminhtml/session')->addError('The profile you are trying to save no longer exists');
    
}
  }

  Mage
::register('current_convert_profile'$profile);

  
$profile->run();
  
  
$batchModel Mage::getSingleton('dataflow/batch');
  if (
$batchModel->getId()) {
    
if ($batchModel->getAdapter()) {
      $batchId 
$batchModel->getId(); 
      
$batchImportModel $batchModel->getBatchImportModel();
      
$importIds $batchImportModel->getIdCollection();  

      
$batchModel Mage::getModel('dataflow/batch')->load($batchId);      
      
$adapter Mage::getModel($batchModel->getAdapter());
      foreach (
$importIds as $importId{
        $recordCount
++;
        
try{
          $batchImportModel
->load($importId);
          if (!
$batchImportModel->getId()) {
             $errors[] 
Mage::helper('dataflow')->__('Skip undefined row');
             continue;
          
}

          $importData 
$batchImportModel->getBatchData();
          try 
{
            $adapter
->saveRow($importData);
          
catch (Exception $e{
            Mage
::log($e->getMessage(),null,$logFileName);          
            continue;
          
}
        
          
if ($recordCount%20 == 0{
            Mage
::log($recordCount ' - Completed!!',null,$logFileName);
          
}
        } 
catch(Exception $ex{
          Mage
::log('Record# ' $recordCount ' - SKU = ' $importData['sku']' - Error - ' $ex->getMessage(),null,$logFileName);        
        
}
      }
      
foreach ($profile->getExceptions() as $e{
        Mage
::log($e->getMessage(),null,$logFileName);          
      
}
      
    }
  }
  
echo 'Import Completed';
  
Mage::log("Import Completed",null,$logFileName);
 
?>

STEP 2: Next step would be to turn on the logging in Admin so that the log file is created.  (Navigation: System->Configuration->Developer->Log Settings)

STEP 3: Test this on your SSH login to check if this works on command line. Use the below command
wget -T0 -t1 -http://<www.example.com>/Cron_Import.php/?files=3XSEEEE.csv

Note here that the file to be imported is passed as parameter in the URL. Susbtitute ‘3XSEEEE.csv’ with your CSV name.

The completion of the above command should create a log file with name <your CSV file name>.log in the var/log directory. Open this file to see the log messages.

STEP 4: Now setup your Cron with the command mentioned in STEP 4. See if the log file gets created once the cron is activated and runs to completion.

Note: You can make it better by ensuring MAIL TO attribute is set for the Cron so that a mail is sent to inform the completion of the cron.
 
Magento Community Magento Community
Magento Community
Magento Community
 
Hadifarnoud
Sr. Member
 
Total Posts:  210
Joined:  2007-10-07
Guildford, Surrey, UK
 

for anyone with Magento 1.6+, not working guys.

I’m looking forward for it as I’m using import for stock updates a lot.

an off topic question, anyone knows what ‘profile history’ tab is for? it’s always empty for me

 
Magento Community Magento Community
Magento Community
Magento Community
 
ergodeit
Jr. Member
 
Total Posts:  7
Joined:  2012-05-02
Richmond, Texas
 

Thanks srinigenie,

I appreciate your help with this posting codes.

I am just checking out the Custom Import/export, AMartinez_CustomImportExport. The set up is to be done the way you scripted the code, right?

Please tell me so I can test it out for my website.

Thanks so much!

 
Magento Community Magento Community
Magento Community
Magento Community
 
CaratCache
Jr. Member
 
Total Posts:  9
Joined:  2009-06-08
 

Using Magento 1.7 and I have tried everything to get a data flow profile to work from cron. Runs without error but doesn’t update the item….

I have an import data profile setup pointing to a file in the import directory. I am able to get this file every hour just fine and run it manually through the profile without error and it updates the product. It is an inventory file from a drop shipper so I am only updating three fields of which I have mapped in the profile.

I have tried every script out there to get his to run as a cron job but none of them update the product. All I need to do is run the profile from a cron job using all of the settings in the profile, though I have tried with and without the settings in the profile and in the scripts. Been at this for two days now and I am at my wits end. Sad thing is I swear this was working yesterday.

I am guessing it has something to do with the profiles only designed to work from a popup, so in the end I guess I just need a way to process this csv file that requires mapped fields to run from cron or simply run the profile as is from cron.

 
Magento Community Magento Community
Magento Community
Magento Community
 
cthomas88
Jr. Member
 
Total Posts:  15
Joined:  2010-03-31
 

@CaratCache have you managed to find a solution to the cron not updating the products in version 1.7?

I have the exact same issue, it works if you do it from the popup but not if you run it from a cron.

Thanks

 
Magento Community Magento Community
Magento Community
Magento Community
 
pancu85
Jr. Member
 
Total Posts:  23
Joined:  2009-05-07
Cesena
 

I have the exact same issue with Magento 1.7 it works if you do it from the popup but not if you run it from a cron.
Have you resolved?

 
Magento Community Magento Community
Magento Community
Magento Community
Magento Community
Magento Community
Back to top
Page 10 of 11