Ok. I’m pretty pumped about this whole DataFlow thing since I use drop ship suppliers pretty much exclusively and all the ones I use provide a data feed of some sort.
I thought I’d struggle through setting up Magento to automatically update my inventory for one specific supplier to get a feel for how it works. I’m gonna need some help for this one…
For this example I’m going to use Computer Gallery ( http://www.compgallery.com/ ) since their data feed is pretty darned good IMHO and there’s not a single product in it that requires any attributes (at least in the traditional sense - size, color, etc.)
The feed I’m looking to use is at http://compgallery.hosting4less.com/ItemDataFiles/Gallery_Complete.csv and is always the same in terms of location and name. Computer Gallery never deletes a product from their feed (even after it’s discontinued and out of stock so we can ignore the situation of what to do if it’s on my site and not in the feed for this one - I’ll hit that one for the next one).
First off, this is my perfect world for how it should work.
1) Get the file via http and save it in a location that I specify on my server. I want to actually save it so I can refer to it should I need to later.
2) Add a field to each product that maps to the supplier name (Computer Gallery in this case) so as to not get products mixed up. Not sure if my logic makes any sense here or not.
3) Import/update all products.
A few notes on #3:
- If a product already exists in my database then it should merely update the quantity in stock. It shouldn’t change any other value. (I believe Magento does this by default - correct me if I’m wrong).
- If a product does not exist in my database it should dump it into a category of my choosing for me to tweak and make live later (I don’t trust any data feeds to import a product and make it live without me first seeing it and ensuring that everything is how I want it (price, image, etc.)
- It would be perfect if it sent me an email for each supplier telling me what products were deactivated due to going out of stock, what products were added new).
4) Of course all of this should run via a cron so I don’t have to touch a darned thing (except for the mentioned tweaking of new products).
5) I’ll add more thing as I think of them…
So first I’d like to focus on #1.
I’m pretty much guessing here using the Actions XML in the default install as a rough guideline.
As a total first attempt rough hack methinks the Actions XML should look something like this:
<action type="varien/http_adapter_curl" method="save"> <var name="type">file</var> <var name="path">http://compgallery.hosting4less.com/ItemDataFiles/Gallery_Complete.csv</var> (quite sure this one can't be right) <var name="filename"><![CDATA[computer_gallery.csv]]></var> </action>
Obviously this gave me an error:
Recoverable Error: Argument 1 passed to Varien_Convert_Action_Abstract::setContainer() must be an instance of Varien_Convert_Container_Interface, instance of Varien_Http_Adapter_Curl given, called in C:\xampp\htdocs\magento\lib\Varien\Convert\Action\Abstract.php on line 158 and defined in C:\xampp\htdocs\magento\lib\Varien\Convert\Action\Abstract.php on line 137 [0] in Varien_Convert_Action_Abstract->setContainer(Varien_Http_Adapter_Curl) in C:\xampp\htdocs\magento\lib\Varien\Convert\Action\Abstract.php on line 158 [1] in Varien_Convert_Action_Abstract->getContainer() in C:\xampp\htdocs\magento\lib\Varien\Convert\Profile\Collection.php on line 140 [2] in Varien_Convert_Profile_Collection->importProfileXml("default") in C:\xampp\htdocs\magento\lib\Varien\Convert\Profile\Collection.php on line 69 [3] in Varien_Convert_Profile_Collection->getProfile("default") in C:\xampp\htdocs\magento\app\code\core\Mage\Core\Model\Convert\Profile.php on line 76 [4] in Mage_Core_Model_Convert_Profile->run() in C:\xampp\htdocs\magento\app\code\core\Mage\Adminhtml\Block\System\Convert\Profile\Run.php on line 66 [5] in Mage_Adminhtml_Block_System_Convert_Profile_Run->toHtml() in C:\xampp\htdocs\magento\app\code\core\Mage\Adminhtml\controllers\System\Convert\ProfileController.php on line 182 [6] in Mage_Adminhtml_System_Convert_ProfileController->runAction() in C:\xampp\htdocs\magento\app\code\core\Mage\Core\Controller\Varien\Action.php on line 325 [7] in Mage_Core_Controller_Varien_Action->dispatch("run") in C:\xampp\htdocs\magento\app\code\core\Mage\Core\Controller\Varien\Router\Admin.php on line 133 [8] in Mage_Core_Controller_Varien_Router_Admin->match(Zend_Controller_Request_Http) in C:\xampp\htdocs\magento\app\code\core\Mage\Core\Controller\Varien\Front.php on line 166 [9] in Mage_Core_Controller_Varien_Front->dispatch() in C:\xampp\htdocs\magento\app\Mage.php on line 380 [10] in Mage::run("base") in C:\xampp\htdocs\magento\index.php on line 28
Fatal error: Call to undefined method Varien_Http_Adapter_Curl::setProfile() in C:\xampp\htdocs\magento\lib\Varien\Convert\Action\Abstract.php on line 140
Am I even close? I’m admittedly a hack when it comes to this but I’m really looking forward to learning as I go here.
This is probably not even close but I’m trying to get a handle on this and banging my head on the keyboard just isn’t cutting it.
Recoverable Error: Argument 1 passed to Varien_Convert_Action_Abstract::setContainer() must be an instance of Varien_Convert_Container_Interface, instance of Varien_Http_Client given, called in C:\xampp\htdocs\magento\lib\Varien\Convert\Action\Abstract.php on line 158 and defined in C:\xampp\htdocs\magento\lib\Varien\Convert\Action\Abstract.php on line 137 [0] in Varien_Convert_Action_Abstract->setContainer(Varien_Http_Client) in C:\xampp\htdocs\magento\lib\Varien\Convert\Action\Abstract.php on line 158 [1] in Varien_Convert_Action_Abstract->getContainer() in C:\xampp\htdocs\magento\lib\Varien\Convert\Profile\Collection.php on line 140 [2] in Varien_Convert_Profile_Collection->importProfileXml("default") in C:\xampp\htdocs\magento\lib\Varien\Convert\Profile\Collection.php on line 69 [3] in Varien_Convert_Profile_Collection->getProfile("default") in C:\xampp\htdocs\magento\app\code\core\Mage\Core\Model\Convert\Profile.php on line 76 [4] in Mage_Core_Model_Convert_Profile->run() in C:\xampp\htdocs\magento\app\code\core\Mage\Adminhtml\Block\System\Convert\Profile\Run.php on line 66 [5] in Mage_Adminhtml_Block_System_Convert_Profile_Run->toHtml() in C:\xampp\htdocs\magento\app\code\core\Mage\Adminhtml\controllers\System\Convert\ProfileController.php on line 182 [6] in Mage_Adminhtml_System_Convert_ProfileController->runAction() in C:\xampp\htdocs\magento\app\code\core\Mage\Core\Controller\Varien\Action.php on line 325 [7] in Mage_Core_Controller_Varien_Action->dispatch("run") in C:\xampp\htdocs\magento\app\code\core\Mage\Core\Controller\Varien\Router\Admin.php on line 133 [8] in Mage_Core_Controller_Varien_Router_Admin->match(Zend_Controller_Request_Http) in C:\xampp\htdocs\magento\app\code\core\Mage\Core\Controller\Varien\Front.php on line 166 [9] in Mage_Core_Controller_Varien_Front->dispatch() in C:\xampp\htdocs\magento\app\Mage.php on line 380 [10] in Mage::run("base") in C:\xampp\htdocs\magento\index.php on line 28
Fatal error: Call to undefined method Varien_Http_Client::setProfile() in C:\xampp\htdocs\magento\lib\Varien\Convert\Action\Abstract.php on line 140
- This specific feed lists cost and MSRP. I take a 33-1/3 margin on this supplier so it needs to take cost X 1.5 to come up with the sale price. I want the products to list MSRP and then a special price.
Example:
Cost: $10
MSRP: $19.95
Selling Price: $15
Couple of things to watch for here:
- If selling price is higher than MSRP then selling price = MSRP.
- Selling price can’t be less than cost (data feeds aren’t perfect)
- Would be nice to be notified if the selling price of a specific product was below a certain margin to give me an opportunity to at least investigate what’s going on.
@Ron: Hi, you can’t use just any class as dataflow container or adapter, it has to extend an abstract and conform to interface.
Let’s see how we create an http adapter. Create lib/Varien/Convert/Adapter/Http/Curl.php:
<?php
// there are many ways in PHP to load remote HTTP file, we'll do curl class Varien_Convert_Adapter_Http_Curl extends Varien_Convert_Adapter_Abstract { // load method public function load() { // we expect <var name="uri">http://...</var> $uri = $this->getVar('uri');
// validate input parameter if (!Zend_Uri::check($uri)) { $this->addException("Expecting a valid 'uri' parameter"); }
// use Varien curl adapter $http = new Varien_Http_Adapter_Curl;
// send GET request $http->write('GET', $uri);
// read the remote file $data = $http->read();
// save contents into container $this->setData($data);
return $this; }
public function save() { // no save implemented return $this; } }
Now you could have dataflow xml like this to save it to local file:
The ultimate idea is for this to be all done automatically via a cron. Problem is that every data feed will be unique so it needs to be handled differently.
There has been a lot of, ummm, silence, from the Magento team about data flow since it was first introduced. Apparently there’s a lot of work going on that will be released soon so it’s probably best to wait and see what they come up with before spending too much time on it IMHO.
I agree. The most simple way should be to create new tables in the database itself with the database fields provided by distributors. The database fields could be filled with a simple script activated by cron job. First download the ftp files from the distributor into a folder on the server. Dispatch the data into the new created database fields. Most of the products need just an update (stock level, price or so). By doing it that way, it would be more secure and with much less lost of speed.
By those steps, the pre-preparation of data would not happen inside the original magento database. After that the insertion could happen inside the whole SQL structure of magento by well preparated data fields.
The first cron job would take a while (depends on the amount of data, pictures a.s.o.). The future updates would be very fast, because only new products would be added or existing products would just get updated by stock level or availability. Actualization is faster than new creation.
I think too, that it would not be a good idea to do automatic deletions of unavailable products, as it happens now by the dataflow procedure in magento. If a product, for example, is not available any more, it could be that customers have already bought some time ago, What happens with the orders if the products are completely deleted from the database? Are those orders still available for the customers and the sellers as order history?
At the moment I have the impression that with the first official release, we will have a new shop system with nice templates but without a professional solution for the products, we would like to sell. The dataflow does not work in the actual state. There is no import for stock level, pictures categories and some other things. If the first release candidate appears, maybe in March, there will not be a professional import tool. As long as the database structure of magento is not as well explained as the template structure, no programmer can develop an import tool. So for big shops with a lot of products, Magento is unusable until a professional import solution is available.
Amen to the statement above. We are planning on having over 500+ products each with warranties drop down menu, and we are looking at days on days of man hours under the current system just getting our data into Magento. We need a better tool to get our data in, and not hearing a thing from the Dev team makes me look bad when I tell our team how we have to get our data into Magento.
We are working on some improvements to the data flow module. One of the things that is taking us a long time is a PHP bug (not a Magento issue) with memory not being cleared after an object is used and unset.
We hope to have it all ready for the next release.
Use this:
Sales: Call 877.832.5289 (N America)|310.295.4144 (International)
