How Import/Export is Implemented in Magento

Last modified by brandondrew on Fri, June 25, 2010 09:41
Source|Old Revisions  

This is an old revision of the document!


Introduction

This long anticipated feature is arguably one of the most important ones in ecommerce world. Flexibility of this interface defines capability of the application to communicate with the outside world, be it legacy systems, 3rd party applications, price aggregators, external order sources, fulfillment systems, accounting, etc.

Thus we took it very seriously and, as with everything else in Magento went one or more steps further than most systems. Effectively any import or export is a conversion. From one medium to another. From database to local file, from FTP server to database, from parsed HTML page to Excel spreadsheet, from Google Spreadsheet to catalog.

Basically, it boils down to loading data from one resource, applying unlimited alterations/reformatting, and saving it to another resource.

Welcome to the world of conversion.

Concept

Let’s take an example of exporting a product catalog to excel spreadsheet file. The logical steps are:

  • Load records from database
  • Convert internal identificators into human understandable text
  • Map database fields to our custom file headings (sku ⇒ Part Number)
  • Format data array as Excel XML
  • Save resulting text as local or remote file

This sequence of actions we call a Profile.

Here we have 4 categories of components:

  • Adapter (input/output)
  • Mapper (data value processing)
  • Parser (data format conversion)
  • Validator (data validation)

Adapters

Adapters are anything that can plug in to external resource. This may include:

  • Local files
  • FTP, SFTP servers
  • Database table
  • Soap services
  • Standard input/output
  • HTTP interface
  • HTML source
  • Google Spreadsheets
  • Custom resources (specialized db interface, cache repository, etc.)

Adapters can have 2 methods:

  • load - extract data from resource into internal container.
  • save - save data into resource.

Mappers

Mappers are for processing data without changing it’s format or structure. They’re useful for tasks such as mapping one field to another or apply formulas on values.

Mappers can run one method: map

Parsers

Parsers are for changing format of data. Examples:

  • CSV text ⇒ Grid array
  • Excel XML text ⇒ Grid array
  • Grid array ⇒ Product model collection

Parsers can have 2 methods:

  • parse - usually convert from more human readable data to machine friendly format
  • unparse - the opposite, into human readable format

You might notice that Parsers are separated from Adapters, when usually they are associated as one thing (export to CSV file).

Validators

Validators are created to make sure that conditions are met. The conditions can be applied to processed data or environment variables.

Validators can only validate.

Implementation

Let’s take an example of exporting catalog products into CSV file.

  1. <action type="catalog/convert_adapter_product" method="load">
  2.     <var name="store">base</var>
  3. </action>
  4.  
  5. <action type="catalog/convert_parser_product" method="unparse">
  6. </action>
  7.  
  8. <action type="varien/convert_mapper_column" method="map">
  9.     <var name="sku">Part#</var>
  10. </action>
  11.  
  12. <action type="core/convert_parser_csv" method="unparse">
  13.     <var name="delimiter">,</var>
  14.     <var name="enclose"><![CDATA["]]></var>
  15.     <var name="fieldnames">true</var>
  16. </action>
  17.  
  18. <action type="core/convert_adapter_io" method="save">
  19.     <var name="type">file</var>
  20.     <var name="path"></var>
  21.     <var name="filename">products.csv</var>
  22.     <var name="link">/dev/moshe/magento/var/export/products.csv</var>
  23. </action>



 

Magento 2 GitHub Repository

Magento Job Board - Some sort of tag line goes here

Latest Posts| View all Jobs