Using SOAP API to delete all products in a category.

Last modified by trogfish on Mon, December 21, 2009 15:44
Source|Old Revisions  

While it is a very, very slow method, an important side benefit of using the SOAP method is that this script can be installed on ANY working webserver, anywhere that Internet access to your Magento website exists. For example, you could load and install a PHP enabled version of Apache, such as the free Community Edition of Zend Server, and repeatedly run this script right from your desktop, without installing or modifying any of these script files on your live Magento installation.

Please note this is my first PHP script of any meaningful size. So, if there are any Guru’s out there who can point out my flaw’s, please do so!

My website and hosting environment created a few challenges.

  1. Many hundreds, or even thousands of products in any given category.
  2. Slow, shared hosting environment.
  3. No Zend on command line, meaning maintenance scripts must use HTTP headers to work within the Zend environment.
  4. Products updated intermittently, but averaging once a month, per brand.

I am using the structure of Brand/Manufacturer = Unique individual category.

When I get product update information, it is all of the brand’s products, all in one master file. Therefore, it is difficult to determine what has been altered vs. deleted vs. added new. Simply over-writing the existing product info can leave deleted products stranded and still active on my website. Very subtle changes, such as a modification of a product description are hard to detect. I have tried running “diff” software on the supplied update files in order to isolate the changes, but the amount of time and errors involved was problematic. Thus, I have taken to periodically deleting all of a brands’ products in a category before re-importing the current products.

Deleting many hundreds or thousands of products in a category also presents some challenges. Using the Magento Admin Back-end in my shared hosting environment, I can only effectively delete about 30-50 products at any given time without issues. (Seems to be a memory management overload issue, and heavy MySQL usage overload creating time-outs, etc..) Attempting to delete many hundreds of products seems to bind up the server such that other tasks and transactions may not complete, or the attempted deletions may not complete.

Manually selecting, and deleting, approximately 30 products over and over and over is time-consuming when hundreds and especially thousands of products are involved. A single system call to delete more, makes the site unresponsive and problematic.

My solution is to delete all products within a category using a cron PHP script. The script uses the Magento SOAP API to simplify being called again and again for hours. It is VERY, VERY, slow, but has the advantage of not over-taxing the site in a modest shared hosting environment.

The first step is to create an API user and Keyword, if you have not created one.

Login to your site’s admin back-end. Browse to System-Web Services and define a new Role set for your API user with the allowed actions you intend to do. You can start by allowing everything for testing, and then scale back to be as restrictive as possible later. Just don’t forget to do so! Save that Role, then navigate to System-Web Services-Users. Add a new User. The user-name is your ‘API USER’ and the key (a password of your choosing) becomes your API Key. Assign the previously created Role to that user, and save.

The API USER/ KEY needs to remain secure. Do not publicly post it, nor leave it in publicly accessible code or files. So, we will use two code fragments. One is hidden from public eyes and contains the API keys and configurations. The second calls the first, but it is publicly viewable, and can be called from any browser. Or, more importantly can be called by “cron” via “wget.”

Put this configuration file in a location NOT accessible from the public. You will need FTP or similar access to your account. Users with Cpanel or other back-end management access can use their web-hosts file management tools, too. For most shared hosting, your publicly viewable Magento web files are stored in a folder called “public_html” or similar. We want to put this configuration file higher in the file tree, so that ONLY YOU, the owner / user of the web-host account can directly access it. I like to put it in a separate folder, to keep it out of the way of other files. I called the file “shredder.inc” in a folder called “myfiles.” Keep note of this location for configuration and future usage.

  1. <?php
  2. //shredder.inc
  3. //magento SOAP category product delete tool
  4. //Keep this file secure.
  5. //WARNING! this tool deletes stuff!
  6. //
  7. //Have you Backed up first?!?
  8. //
  9. // I found inspiration, or stole code from:
  10. // (You might be interested in viewing...)
  11. //http://php.net/manual/en/
  12. //http://blog.chapagain.com.np/php-read-write-xml-with-simplexml/
  13. //http://devzone.zend.com/article/689
  14. //http://www.webdeveloper.com/forum/showthread.php?t=187130
  15. //http://www.webmasterkitchen.com/article/problems-with-php-simplexml-soap-results/
  16. //http://www.magentocommerce.com/boards/viewthread/24801/
  17. //
  18. // Let's get busy...
  19. // Bend me to your will and Configure me:
  20. $mageapiUser='YourSecretApiUserName'; //Your soap user Name
  21. $mageapiKey='YourSecretApiKey'; //Your soap user name's KEY (a password, really.)
  22. $mage_url = 'http://<YourMagentoSite>/api/soap/?wsdl'//Your URL in the form 'http://<your-magento-site>/api/soap/?wsdl'
  23. $comline = 0; // set to 1 (numeral one.) for command line usage. Must have the semicolon present.
  24. $categoryId = 50; // Put here your category id; Be Careful!
  25. $storeId = 1; // Enter your store level number.  Most will use numeral one.
  26. $hm=10; //Whole number.  How many products to delete each time script is called.  Start with 10 and experiment.
  27. ?>

Next, save the main script file in your Magento root folder. This may be within the aforementioned “public_html” or similar. If you have multiple domains, or installed Magento in a subfolder, it may be “public_html/<YourMagentoSite>.” It is not critical where you place this file, but you will have to keep track of it’s location in order to connect the two files together.

  1. <?php
  2. error_reporting(E_ALL);
  3. ini_set("display_errors", 1);
  4. ini_set('max_execution_time', '0'); // 0 = no limit.
  5. ob_implicit_flush(true); //Saves having to flush manually
  6. ignore_user_abort(); //Lets the script run to completion even if you kill the browser   
  7. $time_start = microtime(true);
  8.  
  9. include_once ('<your full path to>/shredder.inc');
  10. //make line breaks easier to deal with...
  11. if ($comline == 1 ){$br = "n";}
  12.     else {$br = "<br/>";}
  13. echo 'Warning!!!! I DELETE STUFF!'.$br;
  14. //make some declarations of love, faith and devotion..
  15. $sku_list=array(); //Working list of Sku's in category
  16. $filters=array()//Filter for making SOAP calls
  17. $skukey='sku'//Working on sku's, not Product Id's.
  18. $deletefilter='product.delete'; //What we are doing to the sku's
  19. // Forward to the webstuff.
  20. $proxy = new SoapClient($mage_url);
  21. echo "Get a session and login:";
  22.     try {$sessionId = $proxy->login($mageapiUser, $mageapiKey);}       
  23.         catch (Exception $e)
  24.     {echo '...ERROR: ', $e->getMessage(), $br; exit();}
  25. echo "...Ready to Rock!".$br;
  26.  
  27. //Get products in listed category
  28. echo "Try to pull in product data for category ".$categoryId." and Store ".$storeId." ";
  29.     try {$assignedProducts = $proxy->call($sessionId, 'category.assignedProducts', array($categoryId, $storeId));}
  30.         catch (Exception $e)
  31.         {echo '...ERROR: ', $e->getMessage(), $br; exit();}
  32.  
  33. if( is_array($assignedProducts) && count($assignedProducts)> 1)
  34.         {echo "...getting array done.".$br;}
  35.         else {echo "Didn't get an array!".$br; exit();}
  36. echo $br;
  37. //Get useable list of SKU's.
  38. $last=count($assignedProducts);
  39. for ($row=0; $row<$last; $row++)
  40.     {$sku_list[$row]=$assignedProducts[$row][$skukey];}
  41. unset($assignedProducts);
  42. //echo" skus: ".$br;var_dump($sku_list);exit(); //For testing.
  43. //
  44. //Get a total count of how many sku's
  45. $last=count($sku_list);
  46. if ($last<1){echo "Don't have any sku's! ".$br;exit();}
  47. $displast=$last-1;
  48. //
  49. ///Be certain before deleting! // This section is for more testing.
  50. //for ($row=0; $row<$last; $row++)
  51. //    {echo "I have sku: ".$sku_list[$row]."  This is ".$row." of ".$displast.$br;}
  52. //
  53. // Start deleting... loop and shred!
  54. if ($hm < $last){$newlast=$last;}
  55. $newlast=$hm;
  56.  
  57. for ($row=0; $row<$newlast; $row++){
  58. echo "Trying to delete sku: ".$sku_list[$row]."  This is number ".$row." of ".$last.$br;
  59. try {$deleteTry = $proxy -> call($sessionId, $deletefilter, $sku_list[$row]);}
  60.         catch (Exception $e)
  61.         {echo '...ERROR: ', $e->getMessage(), $br; exit();}
  62. if ($deleteTry){echo "Success!".$br;}
  63.     else echo "UH.OH. --> Didn't work out!";
  64. sleep(5);//make sure mysql is not overloaded.  You may try to adust.  This number is Seconds.
  65. }
  66. $minutes=0;
  67. $time_end = microtime(true);
  68. $time = $time_end - $time_start;
  69. if ($time > 60 ){$minutes=round($time/60); $time=$time-($minutes*60);}
  70. $time=round($time);
  71. echo "Did ".$newlast." sku's in ".$minutes." minutes and ".$time." seconds".$br;
  72. echo $br.$br."I am done!".$br;
  73. exit();
  74. ?>

Next BACKUP your site and database!

Then, carefully browse your category tree and make note the store Id and category number of the product you wish to delete. Change the relevant variables within the ‘shredder.inc’ file as needed. As a test, call the webpage “http://<YourMagentoSite>/productshredder.php” from a browser and be sure everything is working as desired.

Once you are sure everything is working correctly, move on to the next item of configuration, which is to repeatedly call the “productshredder.php” script. For most users, this will be done from with your web host configuration panel tools. Create a cron entry which calls the productshredder.php page every 5-10 minutes or so. Depending upon your site’s performance, you may wish to make other configuration changes within the “shredder.inc” file too. Your cron entry may look something similar to this: “*/5 * * * 0,1,2,3,4,5,6 wget -q http://<YourMagentoSite>/productshredder.php” If you have trouble with this, browse the web for “cron” and “wget”; or contact your web host as needed. Periodically monitor your progress by viewing your Category Product Tree. Then, go do other things, like read your email, go watch a movie. Go to bed for evening, etc.

EXTREMELY IMPORTANT: Once completed, DELETE your cron job. Otherwise, any products you add back into the category will just get deleted again!

ALSO EXTREMELY IMPORTANT: RENAME the productshredder.php to something that renders it ineffectual until you need to use it again. Rename it to something like “productshredder.backup”. Otherwise, anyone can call the script and delete your products!

Now that you have a clean slate in your category, re-import your current product list, or otherwise do what you may need to do to your site.




 

Magento 2 GitHub Repository

Magento Job Board - Some sort of tag line goes here

Latest Posts| View all Jobs