Magento Forum

   
A suggestion to import configurable products + categories
 
emmenko
Sr. Member
 
Total Posts:  96
Joined:  2010-10-20
 
Quinty - 15 November 2010 06:09 AM

emmenko - 15 November 2010 05:02 AM

I doubt that it’s the “cron” part that doesn’t work, it’s an attribute that can’t get processed. In Magento (core) not at all places those variables or objects are verified before they are called with methods like getId(), that’s what ‘s causing the fatal error.

I discovered Magento recently. It’s a shame these import features aren’t implemented in the core.. So many thanks to amartinez!

Yeah, but how do you explain that doing the import in backend everything works fine, but when the import is called by the cron_import_products.php there is an error?

Maybe isn’t the file cron_import_products itself that causes the problem, but a non-well-set method...in any case the method is called by the cron file, so I think we have to start digging from that one.
I did some “rudimental debug” and it seems that the script stops when it finds the first configurable product, in the method saveRow in the Productimport.php, at this line:

$cspa = $product->getTypeInstance()->getConfigurableAttributesAsArray($product);

But, as I said, the productimport.php works fine, if it’s run in backend...so I thought the issue is in the parameter passed to the method saveRow, which in the cron file (before the calling) is like that:

$importData = $batchImportModel -> getBatchData();

Maybe we have to look at it....I’m not an expert, so I don’t know where to look...I asked amartinez to take a look...I hope he could find the solution soon!

Nicola

 
Magento Community Magento Community
Magento Community
Magento Community
 
emmenko
Sr. Member
 
Total Posts:  96
Joined:  2010-10-20
 

Hi averyone,

Finally I managed to do a clean import with a cron!

I have to thx Quinty e amartinez for the support on the last days…

Anyway, soon amartinez will publish a stable version of his script for magento version 1.4+!!!
I (and we all I think) am looking forward for this…

Bye

Nicola

 
Magento Community Magento Community
Magento Community
Magento Community
 
Quinty
Jr. Member
 
Total Posts:  12
Joined:  2010-11-15
 

An issue I am running into is that the importing goes very slow. It’s fast in the beginning but it accumulates to longer and longer process times for each next row. I will try to figure out what causes this.
My guess is that an array must be reset/unset somewhere.

A lame workaround is to split the data into smaller files, but that is not really a solution.

 
Magento Community Magento Community
Magento Community
Magento Community
 
emmenko
Sr. Member
 
Total Posts:  96
Joined:  2010-10-20
 
Quinty - 22 November 2010 07:16 AM

An issue I am running into is that the importing goes very slow. It’s fast in the beginning but it accumulates to longer and longer process times for each next row. I will try to figure out what causes this.
My guess is that an array must be reset/unset somewhere.

A lame workaround is to split the data into smaller files, but that is not really a solution.

Really??
I’m still testing with few products (like 50 rows)....but it will be a problem if it goes slow!

How many products are you importing?

 
Magento Community Magento Community
Magento Community
Magento Community
 
Curt Maybury
Jr. Member
 
Avatar
Total Posts:  13
Joined:  2007-11-20
 

5000 products takes about 90 minutes on my install, 800 of those are configurable.

However I’ve tweaked a lot to stop the server from indexing automatically during the import, and now have to run my indexing via cron wink

 
Magento Community Magento Community
Magento Community
Magento Community
 
nafnaf1000
Sr. Member
 
Total Posts:  209
Joined:  2008-02-21
 

People....

Time to move away from DataSlow....

I got 4000 item in my store in under 5 seconds.

the quest for an efficient product import script
http://www.magentocommerce.com/boards/viewthread/201210/

 
Magento Community Magento Community
Magento Community
Magento Community
 
emmenko
Sr. Member
 
Total Posts:  96
Joined:  2010-10-20
 
nafnaf1000 - 22 November 2010 08:13 AM

People....

Time to move away from DataSlow....

I got 4000 item in my store in under 5 seconds.


the quest for an efficient product import script
http://www.magentocommerce.com/boards/viewthread/201210/

I know that but there is onle little problem: it doesn’t support Configurable Products...and the store I’m working at has quite every product as configurable + simple.

Do you have any suggestion???

Thx

Nicola

 
Magento Community Magento Community
Magento Community
Magento Community
 
nafnaf1000
Sr. Member
 
Total Posts:  209
Joined:  2008-02-21
 

In all honesty I do not know anything about Configurable Products. In all my time playing around with magneto I only relay looked at simple products. I do know that there are a few people testing configurable…

the more please that test the faster the bugs get fixed. smile

(Some times.)

 
Magento Community Magento Community
Magento Community
Magento Community
 
emmenko
Sr. Member
 
Total Posts:  96
Joined:  2010-10-20
 
nafnaf1000 - 22 November 2010 08:29 AM

In all honesty I do not know anything about Configurable Products. In all my time playing around with magneto I only relay looked at simple products. I do know that there are a few people testing configurable…

the more please that test the faster the bugs get fixed. smile

(Some times.)

Mmmh...I don’t know which kind of shops most people do (because they use only simple products) but the shop I’m creating is a clothes/shoes shop....so I have to use configurable products (e.g. a jacket has more sizes and colors...so I create a configurable product with simple products associated which are the “variants” for the main product...I cannot do otherwise)

But it’s a shame that this kind of import is not well supported by magento....I hope I will not have big issues with slow import once I have the entire catalog.
Otherwise I could split the csv into more little csv...I read that this procedure increses a bit the speed...but it’s not really a solution, especially if I use a cron job. I have in any case an automatic import for each file, which means basically a lot of time....

...uff! :(

Nicola

 
Magento Community Magento Community
Magento Community
Magento Community
 
Curt Maybury
Jr. Member
 
Avatar
Total Posts:  13
Joined:  2007-11-20
 

I’ve had Magmi installed since the first inception - here on the forums, update it constantly.  But it doesn’t play nice with configurables, and until I’ve worked out what I need to do to make them work, I can only use Magmi on the second store, which is simple only.

Magmi is awesome - I write my own data download script which creates a format that Magmi likes from the dropshipper at 5am every day, Magmi runs at 5.05 and the site is never out of date.

But my main store needs configurable options and it seems no-one is interested in SQL based configurable updates / additions at the moment (I’m not surprised, I’ve been working on it for 4 months and had to fall back to heavily modding dataflow to get them working how I want)

90mins compared to the 4 hours it did take to import is an improvement, but still so far away from realistic that I’m considering ditching magento.  Shame, she’s a pretty lil beast with a modern look… and the efficiency of the UK HMRC.

 
Magento Community Magento Community
Magento Community
Magento Community
 
emmenko
Sr. Member
 
Total Posts:  96
Joined:  2010-10-20
 
doofah - 22 November 2010 08:53 AM

I’ve had Magmi installed since the first inception - here on the forums, update it constantly.  But it doesn’t play nice with configurables, and until I’ve worked out what I need to do to make them work, I can only use Magmi on the second store, which is simple only.

Magmi is awesome - I write my own data download script which creates a format that Magmi likes from the dropshipper at 5am every day, Magmi runs at 5.05 and the site is never out of date.

But my main store needs configurable options and it seems no-one is interested in SQL based configurable updates / additions at the moment (I’m not surprised, I’ve been working on it for 4 months and had to fall back to heavily modding dataflow to get them working how I want)

90mins compared to the 4 hours it did take to import is an improvement, but still so far away from realistic that I’m considering ditching magento.  Shame, she’s a pretty lil beast with a modern look… and the efficiency of the UK HMRC.

I quote you!!!!!!!

The thing with configurable is very annoying....I hope someone will find a solution (I’m still a noob with magento, so I cannot give too much help)

 
Magento Community Magento Community
Magento Community
Magento Community
 
nafnaf1000
Sr. Member
 
Total Posts:  209
Joined:  2008-02-21
 

@emmenko

I sell all electronics...I don’t get involved in configurable product at all.

@doofah

Magmi now has a plug-in for file mapping smile So you no longer need to creates a format that Magmi likes....It like what ever file u give it.

I am in the need of what you have to download my file and update my site. care to share Please.

I would love to help out in coding for this configurables but I am no programmer. (I just understand it and how it works)

 
Magento Community Magento Community
Magento Community
Magento Community
 
Curt Maybury
Jr. Member
 
Avatar
Total Posts:  13
Joined:  2007-11-20
 

nafnaf - I’ll send you an example of how I do things - it’s not the most tidy, but my supplier doesn’t offer a feed file in a useful format, just 4 files you have to pull together. 

What i’d like to try is using magmi to import all the simple products, then generating the configurable product from all simple products with a set attribute - say a >8 digit SKU.  MIght try that later on.

Of course my version of this script also downloads images from a remote source and various other tweaks lol - i’d have to cry if they stopped being useful.

 
Magento Community Magento Community
Magento Community
Magento Community
 
Quinty
Jr. Member
 
Total Posts:  12
Joined:  2010-11-15
 

Ok I tried this to check if I wans’t imagining the time increases

<?php
$time_start 
microtime(true);

$adapter -> saveRow$importData );

$time_end microtime(true);
$time $time_end $time_start;
print 
"Saved row $recordCount in $time seconds\n";
?>

output:

Saved row 2 in 1.07412004471 seconds
Saved row 3 in 0.898960113525 seconds
Saved row 4 in 0.953588962555 seconds
Saved row 5 in 0.925822973251 seconds
Saved row 6 in 0.963361024857 seconds
Saved row 7 in 0.965110063553 seconds
Saved row 8 in 0.988428115845 seconds
Saved row 9 in 1.00181698799 seconds

// snip

Saved row 98 in 1.97649097443 seconds
Saved row 99 in 1.99559402466 seconds
Saved row 100 in 1.93208503723 seconds
Saved row 101 in 1.94460606575 seconds
Saved row 102 in 2.21192789078 seconds
Saved row 103 in 1.99672007561 seconds
Saved row 104 in 2.03837108612 seconds
Saved row 105 in 2.01460385323 seconds
Saved row 106 in 2.07052206993 seconds
Saved row 107 in 2.27176094055 seconds

I tried to find some information with xdebug and webgrind + wincachegrind but didn’t find anything useful.
The reason is probably in those logs but I can’t find the increasing functions..
Also, with webgrind I didn’t see function times at all. 100 rows created a log of 2GB so be cautious if you try this. I had better luck opening that with wincachgrind, but webgrind also seems to be a very useful tool.

So for now I think I am going to split the files grin

 
Magento Community Magento Community
Magento Community
Magento Community
 
edwong
Jr. Member
 
Total Posts:  15
Joined:  2010-10-09
 

this method works fine. can import multiple images, configurable products and simple products.
(v 1.4.1.1)

One question on related product:
p_1, sku_1, related_sku_2
p_2, sku_2, related_sku_1

after upload, product 2 has the correct related product.
But product 1 does not, because at the time of upload, product 2 is not yet exist.

Any ideas how to fix this? Thx

Ed

 
Magento Community Magento Community
Magento Community
Magento Community
Magento Community
Magento Community
    Back to top