Posting in the Magento forums has been disabled pending the implementation of a new and improved forum solution which should better serve the community.

For new questions please post at magento.stackexchange.com, the community-run support site for the Magento community. We will be providing updates on the new forum solution soon. For questions or concerns please email community@magento.com.

Magento Forum

Importing products - using a cron job to call Magento Dataflow import
 
BetaJ
Jr. Member
 
Avatar
Total Posts:  7
Joined:  2007-10-08
Hamburg, Germany
 

Hello javiforos,
you have to use all attribute names in english, for example
status="Enabled" or visibility="Catalog, Search”.
Best Regards
BetaJ

 
Magento Community Magento Community
Magento Community
Magento Community
 
javiforos
Jr. Member
 
Total Posts:  5
Joined:  2009-08-21
 
BetaJ - 21 August 2009 05:16 AM

Hello javiforos,
you have to use all attribute names in english, for example
status="Enabled" or visibility="Catalog, Search”.
Best Regards
BetaJ

BetaJ,
it works if I use native magento import popup. Its neccessary to set some configuration before process, I have to check

I also noticed, that in popup process it calls a method to save url in core_url_rewrite table

 
Magento Community Magento Community
Magento Community
Magento Community
 
EazyEggbert
Jr. Member
 
Total Posts:  18
Joined:  2009-02-13
 

Nice, BetaJ....works for me, except for that it does not import numbers, where the decimal character is “,” (comma). Any idea how I set this up? Of course I set my profile to use “,” instead of “.”. I manipulated my file to numbers with “.” decimals and it worked...if I keep “,” it puts the respective number to 999.999999999.

Thank you and best regards,

Felix

 
Magento Community Magento Community
Magento Community
Magento Community
 
javiforos
Jr. Member
 
Total Posts:  5
Joined:  2009-08-21
 
EazyEggbert - 24 August 2009 04:39 AM

Nice, BetaJ....works for me, except for that it does not import numbers, where the decimal character is “,” (comma). Any idea how I set this up? Of course I set my profile to use “,” instead of “.”. I manipulated my file to numbers with “.” decimals and it worked...if I keep “,” it puts the respective number to 999.999999999.

Thank you and best regards,

Felix

decimal character can be set in the profile form

 
Magento Community Magento Community
Magento Community
Magento Community
 
EazyEggbert
Jr. Member
 
Total Posts:  18
Joined:  2009-02-13
 

@javiforos: unfortunately that doesn’t work for me! I put “,” in that field but still same result...any ideas?

 
Magento Community Magento Community
Magento Community
Magento Community
 
BetaJ
Jr. Member
 
Avatar
Total Posts:  7
Joined:  2007-10-08
Hamburg, Germany
 

Hello,
I think you must use a ‘.’ when you doing it via this php-method, because everything must be in the english format.
The question is, how to set up the local settings to a new php file like these, and I don’t know that. Maybe somebody
else?
It must be something like

Mage::app()->setCurrentLocale(....) // or something similar
but I don’t search in the code for that because english with ‘.’ work for me perfect.
Greetings
BetaJ
 
Magento Community Magento Community
Magento Community
Magento Community
 
shane_eller
Jr. Member
 
Total Posts:  4
Joined:  2009-07-16
Indianapolis, Indiana
 

I’ve created a different version of this script to allow you to export products using a cron job…

----------------------

//uncomment when moved to server - to ensure this page is not accessed from anywhere else
//if ($_SERVER['REMOTE_ADDR'] !== '<your server ip address') {
//  die("You are not a cron job!");
//}

require_once 'app/Mage.php';
// wget -O - http://<www.example.com>/Cron_Import.php
  
umask(0);

  
//$_SERVER['SERVER_PORT']='443';
  
Mage::app();

  
$profileId 7//put your profile id here
 
  
Mage::log("Import Started",null,$logFileName);  
 
  
$profile Mage::getModel('dataflow/profile');
  
  
$userModel Mage::getModel('admin/user');
  
$userModel->setUserId(0);
  
Mage::getSingleton('admin/session')->setUser($userModel);
  
  if (
$profileId{
    $profile
->load($profileId);
    if (!
$profile->getId()) {
      Mage
::getSingleton('adminhtml/session')->addError('The profile you are trying to save no longer exists');
    
}
  }

  Mage
::register('current_convert_profile'$profile);

  
$profile->run();
  
  echo 
'Export Completed';
----------------------
 
Magento Community Magento Community
Magento Community
Magento Community
 
abBMS
Jr. Member
 
Total Posts:  5
Joined:  2008-07-16
 
EazyEggbert - 24 August 2009 11:27 PM

@javiforos: unfortunately that doesn’t work for me! I put “,” in that field but still same result...any ideas?

Mage::getSingleton(’core/locale’)->setLocale(’en_US’);

That’s the way I’ve found to solve the same problem.

 
Magento Community Magento Community
Magento Community
Magento Community
 
korb
Jr. Member
 
Avatar
Total Posts:  4
Joined:  2009-10-09
 

Hello guys

As I read posts above, and another threads, it seems my need is a little complicated.

I would like to import product information (name, sku, short description, description, stock information, custom parameters for frontend search, metadata, manufacturer, price and taxe) and product images all in a single file.

I would also want to make possible updating price and stock information with a single file for all products.

Is the code valid for both of my needs, what adjustments should I have to do?

BTW When is a CSV imoprt feature available for Magento?

Thank you

 
Magento Community Magento Community
Magento Community
Magento Community
 
bazen
Jr. Member
 
Total Posts:  8
Joined:  2009-09-17
 

With this Script in the shell the $adapter doesn’t get the Profile-Params.
You have to add this line:

$adapter->setBatchParams($batchModel->getParams());

all the code i’m running:

<?php
//uncomment when moved to server - to ensure this page is not accessed from anywhere else
//if ($_SERVER['REMOTE_ADDR'] !== '<your server ip address') {
//  die("You are not a cron job!");
//}
#echo "hallo";
ini_set('memory_limit''4000M');
ini_set("max_execution_time""30000");
//uncomment when moved to server - to ensure this page is not accessed from anywhere else
//if ($_SERVER['REMOTE_ADDR'] !== '<your server ip address') {
//  die("You are not a cron job!");
//}

require_once 'app/Mage.php';

umask(0);

//$_SERVER['SERVER_PORT']='443';
Mage::app()->setCurrentStore(Mage_Core_Model_App::ADMIN_STORE_ID);;


if(
$argc<3{
    
die("[SYNTAX ERROR]\nSyntax: php import.php profile_id filename.csv");
}

$filename 
$argv[2];
$profileId intval($argv[1]);
Mage::app()->getRequest()->setParam('files',$filename);


if (!isset(
$filename)) {

    
die("No file has been set!");
}
$logFileName
$filename.'.log';
$recordCount 0;

Mage::log("Import Started",null,$logFileName);

$profile Mage::getModel('dataflow/profile');

$userModel Mage::getModel('admin/user');
$userModel->setUserId(0);
Mage::getSingleton('admin/session')->setUser($userModel);

if (
$profileId{
    $profile
->load($profileId);
    if (!
$profile->getId()) {
        Mage
::getSingleton('adminhtml/session')->addError('The profile you are trying to save no longer exists');
    
}
}

Mage
::register('current_convert_profile'$profile);

$profile->run();

$batchModel Mage::getSingleton('dataflow/batch');

print_r($batchModel->debug());

if (
$batchModel->getId()) {
    
if ($batchModel->getAdapter()) {
        $batchId 
$batchModel->getId();
        
$batchImportModel $batchModel->getBatchImportModel();
        
$importIds $batchImportModel->getIdCollection();

        
$batchModel Mage::getModel('dataflow/batch')->load($batchId);
        
$adapter Mage::getModel($batchModel->getAdapter());
        
$adapter->setBatchParams($batchModel->getParams());
        foreach (
$importIds as $importId{
            $recordCount
++;
            try 
{
                $batchImportModel
->load($importId);
                if (!
$batchImportModel->getId()) {
                    $errors[] 
Mage::helper('dataflow')->__('Skip undefined row');
                    continue;
                
}

                $importData 
$batchImportModel->getBatchData();
                try 
{
                    $adapter
->saveRow($importData);
                
catch (Exception $e{
                    
echo $e->getMessage();
                    continue;
                
}

                
if ($recordCount%20 == 0{
                    
echo 'Processed: '.$recordCount ''.chr(13);
                
}
            } 
catch(Exception $ex  ){
                
echo 'Record# ' $recordCount ' - SKU = ' $importData['sku']' - Error - ' $ex->getMessage();
            
}
        }
        
foreach ($profile->getExceptions() as $e{
            
echo $e->getMessage();
        
}

    }
}
echo 'Import Completed';
 
Magento Community Magento Community
Magento Community
Magento Community
 
bazen
Jr. Member
 
Total Posts:  8
Joined:  2009-09-17
 

With this Script in the shell the $adapter doesn’t get the Profile-Params.
You have to add this line:

$adapter->setBatchParams($batchModel->getParams());

all the code i’m running:

<?php
//uncomment when moved to server - to ensure this page is not accessed from anywhere else
//if ($_SERVER['REMOTE_ADDR'] !== '<your server ip address') {
//  die("You are not a cron job!");
//}
#echo "hallo";
ini_set('memory_limit''4000M');
ini_set("max_execution_time""30000");
//uncomment when moved to server - to ensure this page is not accessed from anywhere else
//if ($_SERVER['REMOTE_ADDR'] !== '<your server ip address') {
//  die("You are not a cron job!");
//}

require_once 'app/Mage.php';

umask(0);

//$_SERVER['SERVER_PORT']='443';
Mage::app()->setCurrentStore(Mage_Core_Model_App::ADMIN_STORE_ID);;


if(
$argc<3{
    
die("[SYNTAX ERROR]\nSyntax: php import.php profile_id filename.csv");
}

$filename 
$argv[2];
$profileId intval($argv[1]);
Mage::app()->getRequest()->setParam('files',$filename);


if (!isset(
$filename)) {

    
die("No file has been set!");
}
$logFileName
$filename.'.log';
$recordCount 0;

Mage::log("Import Started",null,$logFileName);

$profile Mage::getModel('dataflow/profile');

$userModel Mage::getModel('admin/user');
$userModel->setUserId(0);
Mage::getSingleton('admin/session')->setUser($userModel);

if (
$profileId{
    $profile
->load($profileId);
    if (!
$profile->getId()) {
        Mage
::getSingleton('adminhtml/session')->addError('The profile you are trying to save no longer exists');
    
}
}

Mage
::register('current_convert_profile'$profile);

$profile->run();

$batchModel Mage::getSingleton('dataflow/batch');

print_r($batchModel->debug());

if (
$batchModel->getId()) {
    
if ($batchModel->getAdapter()) {
        $batchId 
$batchModel->getId();
        
$batchImportModel $batchModel->getBatchImportModel();
        
$importIds $batchImportModel->getIdCollection();

        
$batchModel Mage::getModel('dataflow/batch')->load($batchId);
        
$adapter Mage::getModel($batchModel->getAdapter());
        
$adapter->setBatchParams($batchModel->getParams());
        foreach (
$importIds as $importId{
            $recordCount
++;
            try 
{
                $batchImportModel
->load($importId);
                if (!
$batchImportModel->getId()) {
                    $errors[] 
Mage::helper('dataflow')->__('Skip undefined row');
                    continue;
                
}

                $importData 
$batchImportModel->getBatchData();
                try 
{
                    $adapter
->saveRow($importData);
                
catch (Exception $e{
                    
echo $e->getMessage();
                    continue;
                
}

                
if ($recordCount%20 == 0{
                    
echo 'Processed: '.$recordCount ''.chr(13);
                
}
            } 
catch(Exception $ex  ){
                
echo 'Record# ' $recordCount ' - SKU = ' $importData['sku']' - Error - ' $ex->getMessage();
            
}
        }
        
foreach ($profile->getExceptions() as $e{
            
echo $e->getMessage();
        
}

    }
}
echo 'Import Completed';
 
Magento Community Magento Community
Magento Community
Magento Community
 
bailey86
Member
 
Total Posts:  47
Joined:  2009-03-05
 

Very interested in using this script - have a question.

This script is to run an import profile - and in the profile the file name is sent in as a parameter.  But why can’t the script pull the file name out of the profile parameters?

I think there’s been some really clever work here - and it’s fantastic that we should be able to run profiles from the command line.

I take it that the profiles need to be a typical type - i.e. Entity Type Products - Direction import - Data transfer local/remote server - Type format - Original Magento attribute names in first row - Yes.

All in all - it’s brilliant that srinigenie has been able to pull out the script and the other contributors have fixed bits.

I’ve not tried it yet - but am really hopeful that it works as I need to pull in approx 90,000 records - and the updates need to be automated.

Will report back on how well the script works.

 
Magento Community Magento Community
Magento Community
Magento Community
 
lilmuckers
Jr. Member
 
Total Posts:  15
Joined:  2008-06-18
 

I’ll just leave this here

http://www.magentocommerce.com/extension/2268

 
Magento Community Magento Community
Magento Community
Magento Community
 
jan212
Guru
 
Avatar
Total Posts:  407
Joined:  2008-01-03
 

Hey lilmuckers and all other members,

i just installed lilmuckers Extension through MC in non productive enviroment - let me say thanks for this at first.

The only problem is that the cron isn’t running cause i don’t now how exactly set up your field “Cron Expression” - i am using a standard cron expression without call to a function atm ( */5 * * * * ). That doesn’t work for me - the profile itself is running correctly if i started it manually. The cron logs are empty…

Anyone with a masterplan?

Thanks, Regards,
Jan

 
Magento Community Magento Community
Magento Community
Magento Community
 
lilmuckers
Jr. Member
 
Total Posts:  15
Joined:  2008-06-18
 

The ‘profile log’ is only populated if you use adaptors that support it (this was an addition required for the clent I was working with, and I figured it might be useful for others)

I assume you’ve set up the standard cron for magento. If not this is in the magento wiki. (I’m on the train with my iPhone so it’s hard to get the URL)
the next thing to check is the profile history.
There should be entries appearing in here when the cron runs.

If you’ve installed v0.1.5 then another check is to not have a cron user set and errors will appear in the profile log.

Hope these help, I’ll post more if I think of it when I get home.

 
Magento Community Magento Community
Magento Community
Magento Community
Magento Community
Magento Community
Back to top