Posting in the Magento forums has been disabled pending the implementation of a new and improved forum solution which should better serve the community.

For new questions please post at magento.stackexchange.com, the community-run support site for the Magento community. We will be providing updates on the new forum solution soon. For questions or concerns please email community@magento.com.

Magento Forum

Page 2 of 29
AMartinez_CustomImportExport 1.5 extension: import/export/and more…! 
 
Bing Dicklepuss
Sr. Member
 
Total Posts:  91
Joined:  2010-07-26
 

Hi AMartinez,

I am looking at your csv1 example, test_single_row_style.csv1, and I have a few questions.

Formatting

All field contents are contained in double quotes “

All fields are separated by comma ,

But the column headers are separated by comma-space ,[space]

Must I follow that format exactly?

I see you have under “_category” the value of “TEST COLLECTION,MALICIA”
That is a multi-valued field value, correct?

If so, then you enclose the field in quotes and use the comma as the separator inside the field.  Is that right?

Column Headers

I see some familiar ones like “qty”, “size”, and “price”, but also some strange ones like “_store”, “_type”, “_product_websites”, “_attribute_set” for example. Why do these start with an underscore, unlike the usual column headings?

Strange values

I notice that you have some familiar values, like “Default” (under column heading “_attribute_set” with its leading underscore) and “simple”, but also some strange ones.
Under “status” you have “1” instead of “Enabled” or “Disabled”.  Under “visibility” you have “1” or “4” instead of “Not Visible Individually”, “Search”, “Catalog”, or “Catalog, Search”.  Do I need to use integer representations for this field?  How do I know which values to use?

Multi-Valued Attributes Setup

It seems like in your post #8 above you are saying that the names of any multi-valued fields have to be hard coded by adding them to the array elements in the setValues and getValues functions in Csv1.php.  So if I have a multi-valued field like “shoe_color”, and another like “style” then my setValues and getValues functions would look like this?

$this->_setValues(array('_store''visibility''_category''_associated_sku','shoe_color','style'));

and

$this->_getValues(array('_store''visibility''_category''_associated_sku','shoe_color','style'));

Then I could import shoes that have ‘shoe_color’ values like “green,blue” and ‘style’ like ‘urban,modern,flashy’, right?  Any fields not listed in the arrays will be treated as single value, it seems.

Thank you, AMartinez for working on this extension.  I hope it works for me, as I have been struggling for months with getting Magento imports to work quickly and have not found a solution yet.  I hope you can answer my questions also.  Thank you once again.

 
Magento Community Magento Community
Magento Community
Magento Community
 
tommyf
Jr. Member
 
Total Posts:  7
Joined:  2008-01-21
 

I exported from another 1.5.1 store using magentos export feature to a new one and got the following errors

Fatal error: Uncaught exception \’Mage_Core_Exception\’ with message \’File var/cus
tomimportexport/productsimport.csv contains 100 corrupt records (from a total of
230)
:::: Invalid value for \’description\’ ::::
In Line(s) 1, 4, 7, 10, 13, 15, 18, 20, 22, 24, 26, 28, 30, 34, 38, 40, 42, 44,
46, 48, 51, 54, 109, 110, 158, 159, 160, 161, 168, 173, 174, 175, 181, 182, 183,
184, 188, 189, 191, 193, 194, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213,
214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229,
230

:::: Orphan rows that will be skipped due default row errors ::::
In Line(s) 2, 3, 5, 6, 8, 9, 11, 12, 14, 16, 17, 19, 21, 23, 25, 27, 29, 31, 32,
33, 35, 36, 37, 39, 41, 43, 45, 47, 49, 50, 52, 53
\’ in /magento2/app/Mage.php:550
Stack trace:
#0/magento2/app/code/community/AMa
rtinez/CustomImportExport/shell/run.php(60): Mage::throwException(\’File var/cust
om...\’)
#1 /home/***/public_html/*****.co in /home/*****/public_html/********/magento2/app/Mage.php on line 550
regulato@regulatorrepairs.com [~/public_html/scubagearsales.com/magento2]

Its seems that all the invalid descriptions are on the rows with _super_products_sku\’s but not sure of why…
Not sure what orphan rows is or errors at bottom (i change my site name and directories to***** to hide them

Any Ideas?
Thanks Again

 
Magento Community Magento Community
Magento Community
Magento Community
 
tommyf
Jr. Member
 
Total Posts:  7
Joined:  2008-01-21
 

I exported from another 1.5.1 store using magentos export feature to a new one and got the following errors

Fatal error: Uncaught exception ‘Mage_Core_Exception’ with message ‘File var/cus
tomimportexport/productsimport.csv contains 100 corrupt records (from a total of
230)
:::: Invalid value for ‘description’ ::::
In Line(s) 1, 4, 7, 10, 13, 15, 18, 20, 22, 24, 26, 28, 30, 34, 38, 40, 42, 44,
46, 48, 51, 54, 109, 110, 158, 159, 160, 161, 168, 173, 174, 175, 181, 182, 183,
184, 188, 189, 191, 193, 194, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213,
214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229,
230

:::: Orphan rows that will be skipped due default row errors ::::
In Line(s) 2, 3, 5, 6, 8, 9, 11, 12, 14, 16, 17, 19, 21, 23, 25, 27, 29, 31, 32,
33, 35, 36, 37, 39, 41, 43, 45, 47, 49, 50, 52, 53
‘ in /magento2/app/Mage.php:550
Stack trace:
#0/magento2/app/code/community/AMa
rtinez/CustomImportExport/shell/run.php(60): Mage::throwException(’File var/cust
om...’)
#1 /home/***/public_html/*****.co in /home/*****/public_html/********/magento2/app/Mage.php on line 550
regulato@regulatorrepairs.com [~/public_html/scubagearsales.com/magento2]

Its seems that all the invalid descriptions are on the rows with _super_products_sku’s but not sure of why…
Not sure what orphan rows is or errors at bottom (i change my site name and directories to***** to hide them

Any Ideas?

 
Magento Community Magento Community
Magento Community
Magento Community
 
amartinez
Guru
 
Avatar
Total Posts:  347
Joined:  2008-11-14
Spain
 
tommyf - 03 May 2011 12:47 PM

:::: Orphan rows that will be skipped due default row errors ::::
In Line(s) 2, 3, 5, 6, 8, 9, 11, 12, 14, 16, 17, 19, 21, 23, 25, 27, 29, 31, 32,
33, 35, 36, 37, 39, 41, 43, 45, 47, 49, 50, 52, 53

First, there should be no orphan columns. Please revise your csv, successfully completing each line with commas are necessary: ,,,,,

Regards,

 
Magento Community Magento Community
Magento Community
Magento Community
 
amartinez
Guru
 
Avatar
Total Posts:  347
Joined:  2008-11-14
Spain
 
Bing Dicklepuss - 03 May 2011 11:05 AM

All fields are separated by comma ,

But the column headers are separated by comma-space ,[space]

Must I follow that format exactly?

strictly only need double quotes are the values ​​of type date

Bing Dicklepuss - 03 May 2011 11:05 AM

I see you have under “_category” the value of “TEST COLLECTION,MALICIA”
That is a multi-valued field value, correct?

correct!

Bing Dicklepuss - 03 May 2011 11:05 AM

If so, then you enclose the field in quotes and use the comma as the separator inside the field.  Is that right?

right!

Bing Dicklepuss - 03 May 2011 11:05 AM

Column Headers

I see some familiar ones like “qty”, “size”, and “price”, but also some strange ones like “_store”, “_type”, “_product_websites”, “_attribute_set” for example. Why do these start with an underscore, unlike the usual column headings?

because new internal arrays to process them

Bing Dicklepuss - 03 May 2011 11:05 AM

Strange values

I notice that you have some familiar values, like “Default” (under column heading “_attribute_set” with its leading underscore) and “simple”, but also some strange ones.
Under “status” you have “1” instead of “Enabled” or “Disabled”.  Under “visibility” you have “1” or “4” instead of “Not Visible Individually”, “Search”, “Catalog”, or “Catalog, Search”.  Do I need to use integer representations for this field?  How do I know which values to use?

right! you must use integer values because “status” field e.g. contains ‘1’ -> ‘Enabled’ , ‘2’ -> ‘Disabled’ an so on

you can go to admin->catalog->product and see these listbox, e.g. “visibility” contains four values: “Not Visible Individually”,"Search", “Catalog”, or “Catalog, Search” that means: 1, 2, 3, 4. Please note that some listbox contains a first “none” value that correlates to “1”.

Bing Dicklepuss - 03 May 2011 11:05 AM

Multi-Valued Attributes Setup

It seems like in your post #8 above you are saying that the names of any multi-valued fields have to be hard coded by adding them to the array elements in the setValues and getValues functions in Csv1.php.  So if I have a multi-valued field like “shoe_color”, and another like “style” then my setValues and getValues functions would look like this?

$this->_setValues(array('_store''visibility''_category''_associated_sku','shoe_color','style'));

and

$this->_getValues(array('_store''visibility''_category''_associated_sku','shoe_color','style'));

Then I could import shoes that have ‘shoe_color’ values like “green,blue” and ‘style’ like ‘urban,modern,flashy’, right?  Any fields not listed in the arrays will be treated as single value, it seems.

It’s right again! smile Please remember to name your file: some_name.csv1 (csv1 means import uses my own import adapter)

Regards!

 
Magento Community Magento Community
Magento Community
Magento Community
 
Bing Dicklepuss
Sr. Member
 
Total Posts:  91
Joined:  2010-07-26
 

Thanks, AMartinez for your detailed response.

I think I understand everything except for the reason to use “_store” instead of “store” and so on.

You said it is for the “internal arrays”.  How do I determine which attributes need to have “_” in front of their names?  Normal Magento import uses “store” and you use “_store”.

Thank You

 
Magento Community Magento Community
Magento Community
Magento Community
 
amartinez
Guru
 
Avatar
Total Posts:  347
Joined:  2008-11-14
Spain
 

Take the values ​​of the system itself (bbdd, .xml config files, etc)

It’s simply the new method smile

Regards!

 
Magento Community Magento Community
Magento Community
Magento Community
 
colea
Jr. Member
 
Total Posts:  3
Joined:  2011-04-29
 

I was looking at the config.xml file for your module, and I was wondering what the last section of the global config was doing exactly.

<importexport module="importexport">
    <
import_entities>
        <
catalog_product translate="label">
            <
model_token>customimportexport/import_entity_product</model_token>
            <
label>Products</label>
        </
catalog_product>
        <
customer translate="label">
            <
model_token>importexport/import_entity_customer</model_token>
           <
label>Customers</label>
        </
customer>
    </
import_entities>
    <
import_product_types>
        <
simple>customimportexport/import_entity_product_type_simple</simple>
        <
configurable>customimportexport/import_entity_product_type_configurable</configurable>
        <
virtual>customimportexport/import_entity_product_type_simple</virtual>
        <
grouped>customimportexport/import_entity_product_type_grouped</grouped>
    </
import_product_types>
</
importexport>

It makes sense (I’m assuming this is where you are telling Magento that you want to use your customimportexport module instead of the regular importexport module), but I was wondering about the syntax for this section of the global config, particularly the xml attributes you used (module and translate)

I’ve gone through numerous explanations of the config.xml syntax but I haven’t come up with anything useful.

Also, have you considered adding the functionality to import/export custom attributes and sets?

 
Magento Community Magento Community
Magento Community
Magento Community
 
amartinez
Guru
 
Avatar
Total Posts:  347
Joined:  2008-11-14
Spain
 
colea - 04 May 2011 07:18 AM

I was looking at the config.xml file for your module, and I was wondering what the last section of the global config was doing exactly.

Hi!

These xml configurations are required to apply my own functions and entity definitions (e.g. to create categories, attribs. or to import single row csv’s)

Plese note the “custom” word inside: <model_token>*custom*importexport/import_entity_product</model_token>

colea - 04 May 2011 07:18 AM

Also, have you considered adding the functionality to import/export custom attributes and sets?

Yes, sure! I’m still working to improve & add more functionality like products export, customers I/O, etc

I am glad to receive your suggestions and needs! Please send me them smile

 
Magento Community Magento Community
Magento Community
Magento Community
 
colea
Jr. Member
 
Total Posts:  3
Joined:  2011-04-29
 

Thanks for the quick response.

amartinez - 04 May 2011 07:41 AM

These xml configurations are required to apply my own functions and entity definitions (e.g. to create categories, attribs. or to import single row csv’s)

That’s what I was figuring (those labels are for the drop-down menu of the import/export area I assume) but I was mostly confused by the module attribute on the importexport node (haven’t seen that one used or explained yet).

Also, does your custom import_entities node completely overwrite the default import_entities node (otherwise why do you include a reference to the default customers import entity)?

 
Magento Community Magento Community
Magento Community
Magento Community
 
amartinez
Guru
 
Avatar
Total Posts:  347
Joined:  2008-11-14
Spain
 

not, my functions only ”overload” the core ones

 
Magento Community Magento Community
Magento Community
Magento Community
 
colea
Jr. Member
 
Total Posts:  3
Joined:  2011-04-29
 
amartinez - 04 May 2011 08:45 AM

not, my functions only ”overload” the core ones

That’s what I was assuming, just got confused by the reference to the default customer import entity

<customer translate="label">
    <
model_token>importexport/import_entity_customer</model_token>
    <
label>Customers</label>
</
customer>

Thanks for the help, and keep up the good work wink

 
Magento Community Magento Community
Magento Community
Magento Community
 
amartinez
Guru
 
Avatar
Total Posts:  347
Joined:  2008-11-14
Spain
 

Customers import/export are not implemented yet smile

 
Magento Community Magento Community
Magento Community
Magento Community
 
amartinez
Guru
 
Avatar
Total Posts:  347
Joined:  2008-11-14
Spain
 

HI again!

I am pleased to inform you that \”AMartinez_CustomImportExport\" extension can now export as well.

Please note that there\’s some \"parameters\" improvement.

It\’s the first step to future functionalities.

Feel free to test it and report any issue or suggestion!

 
Magento Community Magento Community
Magento Community
Magento Community
 
fear4me
Jr. Member
 
Total Posts:  18
Joined:  2008-07-08
 

@AMartinez: Thanks for your great contribution! I was looking for it for a while and programmed some different import scripts (SOAP, dump and using the profiles) but sometimes it took more than 24 hours to complete… so the next import would be waiting again when the other finishes. Yesterday I did an import and finished in about 22 minutes for 50.000 products! Awesome! However… today I’m trying the second run of the import (to test updates) where I have again a full CSV with 50.000 products. But somehow the script keeps running at “Catalog URL Rewrites”. When I look with “top” I still have a PHP process running with CPU consumption. When I look in phpMyAdmin there is executed a query just once in about 10 secs? on the core_url_rewrite table. The script is already running for more than 2 hours now. Any idea what could cause the problem?

p.s.... I’m running the MagentoConnect version 005 of your script.

 
Magento Community Magento Community
Magento Community
Magento Community
Magento Community
Magento Community
Back to top
Page 2 of 29