Posting in the Magento forums has been disabled pending the implementation of a new and improved forum solution which should better serve the community.

For new questions please post at magento.stackexchange.com, the community-run support site for the Magento community. We will be providing updates on the new forum solution soon. For questions or concerns please email community@magento.com.

Magento Forum

AMartinez_CustomImportExport 1.5 extension: import/export/and more…! 
 
neilyw
Jr. Member
 
Total Posts:  4
Joined:  2010-01-06
 

Hi, thanks for what seems to be a great and much needed extension.  However, I’m getting the following error when I try running any of your examples:
Parse error: syntax error, unexpected T_STRING, expecting T_OLD_FUNCTION or T_FUNCTION or T_VAR or ‘}’ in /.../magento/app/code/community/AMartinez/CustomImportExport/shell/run.php on line 23.

I’m using Magento ver. 1.5.1.0 and the latest release of the extension.

 
Magento Community Magento Community
Magento Community
Magento Community
 
niglesi
Jr. Member
 
Total Posts:  15
Joined:  2011-03-23
 
amartinez - 07 July 2011 08:51 AM

NEW 014 RELEASE

Now you can import images and media galleries by using both methods: own one and magento (that shows in admin, etc etc)

You can read further instructions here.

I would appreciate your comments!

El mejor comentario que puedo darte es que el el script más útil de todos los existentes.
Que tu esfuerzo nos ha ayudado a muchos.
La simplicidad de lo que hace, me ha ayudado a perder le miedo a ciertas partes del magento.

Muchas gracias y adelante.

 
Magento Community Magento Community
Magento Community
Magento Community
 
kachkatos00
Jr. Member
 
Total Posts:  4
Joined:  2011-04-17
 

get this error:

php -f amartinez_customimportexport.php—-e

Start products output to file var/customimportexport/products.csv
<br />
<b>Fatal error</b>:  Allowed memory size of 268435456 bytes exhausted (tried to allocate 43 bytes) in <b>/html/magento151/lib/Varien/Object.php</b> on line <b>215</b><br />

Magento Version 1.5.1.0
1.5.014

 
Magento Community Magento Community
Magento Community
Magento Community
 
infrid
Member
 
Total Posts:  42
Joined:  2011-05-12
 
amartinez - 11 July 2011 06:04 AM

Please try to delete all non-root categories, and import again.

You can use php -f amartinez_customimportexport.php—-deleteallproducts (caution: all products and non-root categories will be deleted) then revise your tables to make sure all are deleted ok

I’ve still the same problem of gaggo, I’ve tried the --deleteallproduct thing, checked the database (and it delete all records) but the script doesn’t still create categories…

The database after the import is similar to th gaggo’s one

Image Attachments
magento_db_categorie.png
 
Magento Community Magento Community
Magento Community
Magento Community
 
itaca
Jr. Member
 
Total Posts:  6
Joined:  2010-10-11
 

Hi, thanks for a great extension.  However, I’m getting the following error when I try to install it by MagentoConnect Manager

No releases available for package “connect.magentocommerce.com/community/AMartinez_CustomImportExport”
Cannot initialize ‘channel://connect.magentocommerce.com/community/AMartinez_CustomImportExport’, invalid or missing package file
Install Errors
Package “channel://connect.magentocommerce.com/community/AMartinez_CustomImportExport” is not valid
PEAR ERROR: install failed

tank for Reply

 
Magento Community Magento Community
Magento Community
Magento Community
 
amartinez
Guru
 
Avatar
Total Posts:  347
Joined:  2008-11-14
Spain
 
neilyw - 25 July 2011 05:05 AM

Hi, thanks for what seems to be a great and much needed extension.  However, I’m getting the following error when I try running any of your examples:
Parse error: syntax error, unexpected T_STRING, expecting T_OLD_FUNCTION or T_FUNCTION or T_VAR or ‘}’ in /.../magento/app/code/community/AMartinez/CustomImportExport/shell/run.php on line 23.

I’m using Magento ver. 1.5.1.0 and the latest release of the extension.

mmm sounds like you invoke script with wrong php command. Please try: whereis php so see your php paths

regards!

 
Magento Community Magento Community
Magento Community
Magento Community
 
amartinez
Guru
 
Avatar
Total Posts:  347
Joined:  2008-11-14
Spain
 
niglesi - 26 July 2011 05:42 AM

amartinez - 07 July 2011 08:51 AM
NEW 014 RELEASE
Now you can import images and media galleries by using both methods: own one and magento (that shows in admin, etc etc)
You can read further instructions here.
I would appreciate your comments!

El mejor comentario que puedo darte es que el el script más útil de todos los existentes.
Que tu esfuerzo nos ha ayudado a muchos.
La simplicidad de lo que hace, me ha ayudado a perder le miedo a ciertas partes del magento.

Muchas gracias y adelante.

Muchísimas gracias por tus palabras de apoyo, intentaré seguir al pié del cañón aportando a la comunidad wink

 
Magento Community Magento Community
Magento Community
Magento Community
 
amartinez
Guru
 
Avatar
Total Posts:  347
Joined:  2008-11-14
Spain
 
kachkatos00 - 27 July 2011 02:10 AM

get this error:

php -f amartinez_customimportexport.php—-e

Start products output to file var/customimportexport/products.csv
<br />
<b>Fatal error</b>:  Allowed memory size of 268435456 bytes exhausted (tried to allocate 43 bytes) in <b>/html/magento151/lib/Varien/Object.php</b> on line <b>215</b><br />

Magento Version 1.5.1.0
1.5.014

try increasing your php memory limit: ini_set(’memory_limit’, ‘1024M’);

try exporting by selecting product blocks through magento admin - import/export

if only for “backup” try using -db option instead

regards!

 
Magento Community Magento Community
Magento Community
Magento Community
 
amartinez
Guru
 
Avatar
Total Posts:  347
Joined:  2008-11-14
Spain
 
itaca - 28 July 2011 09:20 AM

Hi, thanks for a great extension.  However, I’m getting the following error when I try to install it by MagentoConnect Manager

No releases available for package “connect.magentocommerce.com/community/AMartinez_CustomImportExport”
Cannot initialize ‘channel://connect.magentocommerce.com/community/AMartinez_CustomImportExport’, invalid or missing package file
Install Errors
Package “channel://connect.magentocommerce.com/community/AMartinez_CustomImportExport” is not valid
PEAR ERROR: install failed

tank for Reply

please go magentoconnect (http://www.magentocommerce.com/magento-connect/amartinez/extension/6513/amartinez_customimportexport) and use “GET EXTENSION KEY” button, select 2.0, then open yout site magentoconnect (http://youdomain.com/downloader) and put this code: community/AMartinez_CustomImportExport

I attach screenshoot

Image Attachments
Captura de pantalla 2011-07-29 a las 17.22.24.png
 
Magento Community Magento Community
Magento Community
Magento Community
 
fedegomez78
Jr. Member
 
Total Posts:  5
Joined:  2011-01-31
 

The best magento import and export extension! so thanks @AMartinez

I\’m getting this error:

:::: Attribute with this code is not super ::::
In Line(s) 5, 14, 19, 24, 33, 38, 43, 48, 52, 61

please take a look at the CSV attached,

File Attachments
products.zip  (File Size: 2KB - Downloads: 56)
 
Magento Community Magento Community
Magento Community
Magento Community
 
webmasterAC
Jr. Member
 
Total Posts:  6
Joined:  2011-06-08
 

Hi Antonio,

I’m new to Magento - developing first project. Your extension looks and feels great. I’m trying to adopt it.
My eshop is going to have 50 000+ items (or maybe even about 100 000).
Imports runs swiftly (good job).
But I have some problems when it comes to 10000 and more in one run.
Got this error:
PHP Fatal error: Uncaught exceprion ‘Exception’ with message ‘Notice: unserialize(): Errror at offset 1677663 of 16777215 bytes in .... app\code\core\MAge\ImportExport\Model\Mysql4\Import\Data.php on line 123’ in ... app\code\core\Mage\Core\funtions:245

So I did it in 10 000 chunks and it worked.
However, when db had about 40 000 and more records, it was imposible to import a single item.
Start products ‘replace’ action from var/customimportexport/testimport3.csv

Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 1200 bytes) in D:\www\ALD\_res\Magento\lib\Varien\Object.php on line 484

Call Stack:
0.0134 52560 1. {main}() D:\www\ALD\_res\Magento\customimport.php:0
2.7623 242528 2. require_once(’D:\www\ALD\_res\Magento\app\code\community\AMartinez\CustomImportExport\shell\run.php’) D:\www\ALD\_res\Magento\customimport.php:15
5.0228 7359168 3. AMartinez_CustomImportExport->run() D:\www\ALD\_res\Magento\app\code\community\AMartinez\CustomImportExport\shell\run.php:661
97.0370 48004048 4. Mage_ImportExport_Model_Import->importSource() D:\www\ALD\_res\Magento\app\code\community\AMartinez\CustomImportExport\shell\run.php:133
97.0413 48004800 5. Mage_ImportExport_Model_Import_Entity_Abstract->importData() D:\www\ALD\_res\Magento\app\code\core\Mage\ImportExport\Model\Import.php:275
97.0413 48004800 6. Mage_ImportExport_Model_Import_Entity_Product->_importData() D:\www\ALD\_res\Magento\app\code\core\Mage\ImportExport\Model\Import\Entity\Abstract.php:523
97.5285 48670744 7. Mage_ImportExport_Model_Import_Entity_Product_Type_Configurable->saveData() D:\www\ALD\_res\Magento\app\code\core\Mage\ImportExport\Model\Import\Entity\Product.php:323
97.5326 48678080 8. Mage_ImportExport_Model_Import_Entity_Product_Type_Configurable->_loadSkuSuperAttributeValues() D:\www\ALD\_res\Magento\app\code\core\Mage\ImportExport\Model\Import\Entity\Product\Type\Configurable.php:365
97.5576 48717536 9. Varien_Data_Collection->getIterator() D:\www\ALD\_res\Magento\lib\Varien\Data\Collection.php:0
97.5576 48717584 10. Mage_Eav_Model_Entity_Collection_Abstract->load(???, ???) D:\www\ALD\_res\Magento\lib\Varien\Data\Collection.php:729
136.7041 210584920 11. Mage_Catalog_Model_Resource_Eav_Mysql4_Product_Collection->_afterLoad() D:\www\ALD\_res\Magento\app\code\core\Mage\Eav\Model\Entity\Collection\Abstract.php:816
149.5411 210585632 12. Mage::dispatchEvent(???, ???) D:\www\ALD\_res\Magento\app\code\core\Mage\Catalog\Model\Resource\Eav\Mysql4\Product\Collection.php:387
149.5411 210585960 13. Mage_Core_Model_App->dispatchEvent(???, ???) D:\www\ALD\_res\Magento\app\Mage.php:416
149.5414 210591128 14. Mage_Core_Model_App->_callObserverMethod(???, ???, ???) D:\www\ALD\_res\Magento\app\code\core\Mage\Core\Model\App.php:1253
149.5414 210591504 15. Mage_CatalogInventory_Model_Observer->addStockStatusToCollection(???) D:\www\ALD\_res\Magento\app\code\core\Mage\Core\Model\App.php:1272
149.5418 210593688 16. Mage_CatalogInventory_Model_Stock_Status->addStockStatusToProducts(???, ???, ???) D:\www\ALD\_res\Magento\app\code\core\Mage\CatalogInventory\Model\Observer.php:112
163.2435 268143824 17. Varien_Object->getData(???, ???) D:\www\ALD\_res\Magento\app\code\core\Mage\CatalogInventory\Model\Stock\Status.php:484

Looks like db size requires more and more memory while importing.

/**
* Initialize existent product SKUs.
*
* @return Mage_ImportExport_Model_Import_Entity_Product
*/
protected function _initSkus()
{
foreach (Mage::getResourceModel(’catalog/product_collection’) as $product) {
$typeId = $product->getTypeId();
$sku = $product->getSku();
$this->_oldSku[$sku] = array(
‘type_id’ => $typeId,
‘attr_set_id’ => $product->getAttributeSetId(),
‘entity_id’ => $product->getId(),
‘supported_type’ => isset($this->_productTypeModels[$typeId])
);
}
return $this;
}

Google searching gave no results about this.
Do you have any clue? Any help is highly appreciated.

Thanks for great extension.
Alex.

 
Magento Community Magento Community
Magento Community
Magento Community
 
Bing Dicklepuss
Sr. Member
 
Total Posts:  91
Joined:  2010-07-26
 
webmasterAC - 12 August 2011 04:56 AM

Hi Antonio,

I’m new to Magento - developing first project. Your extension looks and feels great. I’m trying to adopt it.
My eshop is going to have 50 000+ items (or maybe even about 100 000).
Imports runs swiftly (good job).
But I have some problems when it comes to 10000 and more in one run.
Got this error:
PHP Fatal error: Uncaught exceprion ‘Exception’ with message ‘Notice: unserialize(): Errror at offset 1677663 of 16777215 bytes in .... app\code\core\MAge\ImportExport\Model\Mysql4\Import\Data.php on line 123’ in ... app\code\core\Mage\Core\funtions:245

So I did it in 10 000 chunks and it worked.
However, when db had about 40 000 and more records, it was imposible to import a single item.
Start products ‘replace’ action from var/customimportexport/testimport3.csv

Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 1200 bytes) in D:\www\ALD\_res\Magento\lib\Varien\Object.php on line 484
...

Looks like db size requires more and more memory while importing.
...

Google searching gave no results about this.
Do you have any clue? Any help is highly appreciated.

Thanks for great extension.
Alex.

Hi Alex, I think this is a very common problem you are having.  I’ve had it myself a several times.

Let me see if I can remember how I fixed it.  I time I just used

ini_set('memory_limit','16M')
in index.php (increasing the memory limit as needed).  I think that magento has it commented out there by default (so that your default php memory settings should work), but you can just uncomment it and change it to a higher number and it should override the global php settings.  I think you said you increased the memory limit already, but can’t you make it larger?
 
Magento Community Magento Community
Magento Community
Magento Community
 
Bing Dicklepuss
Sr. Member
 
Total Posts:  91
Joined:  2010-07-26
 
amartinez - 20 July 2011 12:09 AM

no: your csv is not correct…

please add your main image to _media_image column too, if you want to link it on admin

your csv appears as:

"sku","_store","_product_websites","_attribute_set","_type","image","small_image","thumbnail","_media_image","_media_attribute_id","_media_label","_media_position","_media_is_disabled"
"TESTSKU1"
,,"base","Lighting","simple","/testsku1_a.jpg","/testsku1_a.jpg","/testsku1_a.jpg","/testsku1_a.jpg,/testsku1_b.jpg,/testsku1_c.jpg,/testsku1_d.jpg","73,73,73,73",,,

please try ABOVE csv

regards!

Thanks, Mr. Martinez.  That did work after all.  I also had to make sure my file was in csv1 format to import multiple gallery images.

Let me also add some things to your example above, since I did a lot of work experimenting before I got things right:

"sku","_store","_product_websites","_attribute_set","_type","image","small_image","thumbnail","_media_image","_media_attribute_id","_media_label","_media_position","_media_is_disabled"
"TESTSKU1"
,,"base","Lighting","simple","/testsku1_a.jpg","/testsku1_a.jpg","/testsku1_a.jpg","/testsku1_a.jpg,/testsku1_b.jpg,/testsku1_c.jpg,/testsku1_d.jpg","73,73,73,73","Test Image 1,Test Image 2,Test Image 3,Test Image 4","1,2,3,4","1,0,0,0"

So, all I did there was add the values for “_media_label”,"_media_position”,"_media_is_disabled"
The first two of those are definitely not required, but you will have images with no label and their order will be random.  All you have to do is include the same number of values (in this case 4 values) in each of those fields to correspond with the images.
I’m not sure if “_media_is_disabled” will default to 0 or to NULL or what.  For me, I use the value “1” for the main image, because I don’t want it to appear in the image gallery (it will still appear in the admin, with the “Exclude” checkbox selected), and the other images with value “0” so that they do appear.

Thanks again for a great script.  I recommend your script over all others for speed and image handling at this point.

Also, again I have to commend you on the image handling.  I’m not sure if it’s you, or magento’s new import system, but either way it is a vast improvement over dataflow.  We probably have 3-10 copies of every image on our server because dataflow would reprocess images every time you ran an import with the image field specified.  Your script does not reprocess the images, and deletes and replaces (at least with the default “replace” action) the existing database image references.  In my tests, I imported and reimported a file with 700 items and about 1300 image references in less than 30 seconds each time.  Actually it was probably less than 15 seconds.  And no duplicates!

Oh and one more thing.  For people importing related items, you can use “_links_related_position” to set the order of display for the items listed in “_links_related_sku” in exactly the same way as described for gallery image positioning above.  It’s very useful if you want the most closely-related items shown first.

 
Magento Community Magento Community
Magento Community
Magento Community
 
webmasterAC
Jr. Member
 
Total Posts:  6
Joined:  2011-06-08
 
Bing Dicklepuss - 12 August 2011 08:14 AM

webmasterAC - 12 August 2011 04:56 AM


Hi Alex, I think this is a very common problem you are having.  I’ve had it myself a several times.

Let me see if I can remember how I fixed it.  I time I just used
ini_set('memory_limit','16M')
in index.php (increasing the memory limit as needed).  I think that magento has it commented out there by default (so that your default php memory settings should work), but you can just uncomment it and change it to a higher number and it should override the global php settings.  I think you said you increased the memory limit already, but can’t you make it larger?

Hi Bing Dicklepuss,

thanks for enlightening me.
I’ve changed memory setting in php.ini but with no result. But there’s also .htaccess setting, which is preferred, so increasing memory limit in it solved my problem with importing more records to big db.

But ‘userialized’ problem is still there (can’t import more them 10K of records in one run). Even 1024M didn’t help.
Any ideas?

Thanks for support.
Alex

 
Magento Community Magento Community
Magento Community
Magento Community
 
Paulus
Guru
 
Total Posts:  335
Joined:  2008-07-21
 

Must have been asked before, but i still cannot find it;/

You wrote:

“Also, add a new adapter “CSV1” that allows to use the old csv files (comma-separated, single line) in the new import system (multi-line-attribute) ”

But how do I export everything first in single lines?

This line export everything in great detail:

php -./amartinez_customimportexport.php -- -e

Thanks!

 
Magento Community Magento Community
Magento Community
Magento Community
Magento Community
Magento Community
Back to top