Try the Demo

Magento

Magento Forum

   
Page 1 of 2
PHP script to refresh magento caches periodically. Catalog Rewrites problems in 1.3+ version? 
 
amartinez
Guru
 
Avatar
Total Posts:  347
Joined:  2008-11-14
Spain
 

Hi, I do this little script running it via cron job like:

00  0 * * *   root /var/www/html/magento/cron_refresh_cache.php >> /var/log/cron_refresh_cache.log

It refresh all magento caches periodically, optimizing it after other tasks, prices or image changes (I do it via automatized tasks too)

But with magento versions 1.3 and above the “Catalog Rewrite” section fails, and I don’t know why.

Please feel free to use try my script. I would appreciate any type of help on the commented error.

Regards!

Antonio

#!/usr/bin/php
<?
/**
* cron_refresh_cache.php
* 
* @copyright  copyright (c) 2009 toniyecla[at]gmail.com
* @license    http://opensource.org/licenses/osl-3.0.php open software license (OSL 3.0)
*/

( !$_SERVER["HTTP_USER_AGENT"] ) or die ( "Nothing to do\n" ); // to run via local browser use ($_SERVER["SERVER_ADDR"] == $_SERVER["REMOTE_ADDR"])

require_once 'app/Mage.php';
umask( 0 );
Mage :: app( "default" );
$ver = Mage :: getVersion();
$userModel = Mage :: getModel( 'admin/user' );
$userModel -> setUserId( 0 );
Mage :: getSingleton( 'admin/session' ) -> setUser( $userModel );

echo "Refreshing cache...\n";
Mage :: app() -> cleanCache();
$enable = array();
foreach ( Mage :: helper( 'core' ) -> getCacheTypes() as $type => $label ) {
    $enable[$type] = 1;
    } 
Mage :: app() -> saveUseCache( $enable );

if ( $ver >= 1.3 ) {
    try {
        Mage :: getResourceModel( 'catalog/category_flat' ) -> rebuild();
        echo "  Flat Catalog Category was rebuilt successfully\n";
        } 
    catch ( Exception $e ) {
        echo $e -> getMessage() . "\n";
        } 

    try {
        Mage :: getResourceModel( 'catalog/product_flat_indexer' ) -> rebuild();
        echo "  Flat Catalog Product was rebuilt successfully\n";
        } 
    catch ( Exception $e ) {
        echo $e -> getMessage() . "\n";
        } 
    }
else {
    try {
        Mage :: getSingleton( 'catalog/url' ) -> refreshRewrites();
        echo "  Catalog Rewrites were refreshed successfully\n";
        }
    catch ( Exception $e ) {
        echo $e -> getMessage() . "\n";
        }
    }

try {
    Mage :: getModel( 'catalog/product_image' ) -> clearCache();
    echo "  Image cache was cleared successfully\n";
    } 
catch ( Exception $e ) {
    echo $e -> getMessage() . "\n";
    } 

try {
    $flag = Mage :: getModel( 'catalogindex/catalog_index_flag' ) -> loadSelf();
    if ( $flag -> getState() == Mage_CatalogIndex_Model_Catalog_Index_Flag :: STATE_RUNNING ) {
        $kill = Mage :: getModel( 'catalogindex/catalog_index_kill_flag' ) -> loadSelf();
        $kill -> setFlagData( $flag -> getFlagData() ) -> save();
        } 
    $flag -> setState( Mage_CatalogIndex_Model_Catalog_Index_Flag :: STATE_QUEUED ) -> save();
    Mage :: getSingleton( 'catalogindex/indexer' ) -> plainReindex();
    echo "  Layered Navigation Indices were refreshed successfully\n";
    } 
catch ( Exception $e ) {
    echo $e -> getMessage() . "\n";
    } 

try {
    Mage :: getSingleton( 'catalogsearch/fulltext' ) -> rebuildIndex();
    echo "  Search Index was rebuilded successfully\n";
    } 
catch ( Exception $e ) {
    echo $e -> getMessage() . "\n";
    } 

try {
    Mage :: getSingleton( 'cataloginventory/stock_status' ) -> rebuild();
    echo "  CatalogInventory Stock Status was rebuilded successfully\n";
    } 
catch ( Exception $e ) {
    echo $e -> getMessage() . "\n";
    } 

echo "Done\n";

?>

File Attachments
cron_refresh_cache.php  (File Size: 3KB - Downloads: 672)
 Signature 

amartinez {web/developer/designer} free modules: CustomImportExport Like it? Donate!

 
 
Posted: May 26 2009
   |  top
 
Magento Community Magento Community
Magento Community
Magento Community
 
amartinez
Guru
 
Avatar
Total Posts:  347
Joined:  2008-11-14
Spain
 

I upload all files to this wiki: http://www.magentocommerce.com/wiki/how_to_automatically_import_simple_grouped_and_configurable_products

s2,

 Signature 

amartinez {web/developer/designer} free modules: CustomImportExport Like it? Donate!

 
 
Posted: May 27 2009
   |  top  |  # 1 
 
Magento Community Magento Community
Magento Community
Magento Community
 
F. P.
Member
 
Total Posts:  55
Joined:  2007-12-09
 

Hi,

Your import script is working great. So thank you for developing and providing that script. There is just one thing to mention: How to import manufacturers with that script. Is there any possibility to do so? As far as I can see, the eav_attribute entity for that purpose is 66.

 
 
Posted: May 27 2009
   |  top  |  # 2 
 
Magento Community Magento Community
Magento Community
Magento Community
 
amartinez
Guru
 
Avatar
Total Posts:  347
Joined:  2008-11-14
Spain
 
F. P. - 27 May 2009 12:33 AM

Hi,

Your import script is working great. So thank you for developing and providing that script. There is just one thing to mention: How to import manufacturers with that script. Is there any possibility to do so? As far as I can see, the eav_attribute entity for that purpose is 66.

Thanks for your comments, please refer the original post for import script at http://www.magentocommerce.com/boards/viewthread/35865/

As I write before, there’s possible to add any field or parameter. You only need to insert some like:

if ($write->fetchOne( "select * from $table where attribute_id=SOME and value_id=SOME") ) {
  $write->query("update $table set attribute_id=SOME and value_id=SOME and value=SOME");
} else {
  $write->query( "insert into $table (attribute_id, value_id, value) values (SOME, SOME, SOME)" );
}

Can you edit the source code and post it here? We would thank you wink

Regards!

Antonio

 Signature 

amartinez {web/developer/designer} free modules: CustomImportExport Like it? Donate!

 
 
Posted: May 27 2009
   |  top  |  # 3 
 
Magento Community Magento Community
Magento Community
Magento Community
 
Skarniva
Jr. Member
 
Total Posts:  16
Joined:  2008-12-23
Tucker, GA
 

Hi amartinez,

Do you know if this script (Catalog Refresh) work on the latest 1.3.2.1?

I haven’t been able to get it to work. I seriously need to rebuild my search index. It won’t work manually in the admin because the global setting on my server has a connection timeout of 60. And I have 100,000 products on my site.

Any help will be sincerely appreciated.

Thanks.

 Signature 

Cheap Office supplies

 
 
Posted: June 23 2009
   |  top  |  # 4 
 
Magento Community Magento Community
Magento Community
Magento Community
 
amartinez
Guru
 
Avatar
Total Posts:  347
Joined:  2008-11-14
Spain
 
Skarniva - 23 June 2009 10:26 AM

Hi amartinez,

Do you know if this script (Catalog Refresh) work on the latest 1.3.2.1?

I haven’t been able to get it to work. I seriously need to rebuild my search index. It won’t work manually in the admin because the global setting on my server has a connection timeout of 60. And I have 100,000 products on my site.

Any help will be sincerely appreciated.

Thanks.

Hi!

I imported 5000 products fine last night.

I think that you need to do the cron job by direct run via ssh connect (like crontab) to avoid the php timeout.

I will talk you later…

Regards!

 Signature 

amartinez {web/developer/designer} free modules: CustomImportExport Like it? Donate!

 
 
Posted: June 25 2009
   |  top  |  # 5 
 
Magento Community Magento Community
Magento Community
Magento Community
 
edmondscommerce
Guru
 
Avatar
Total Posts:  342
Joined:  2008-08-26
 

nice work grin

 Signature 

Edmonds Commerce
+44 (0)1274 590036
Expert UK Based
Magento and Magento Enterprise Web Development

 
 
Posted: June 25 2009
   |  top  |  # 6 
 
Magento Community Magento Community
Magento Community
Magento Community
 
amartinez
Guru
 
Avatar
Total Posts:  347
Joined:  2008-11-14
Spain
 

I’’m glad to be helpful smile

edmondscommerce - 25 June 2009 02:04 AM

nice work grin

 Signature 

amartinez {web/developer/designer} free modules: CustomImportExport Like it? Donate!

 
 
Posted: June 25 2009
   |  top  |  # 7 
 
Magento Community Magento Community
Magento Community
Magento Community
 
amartinez
Guru
 
Avatar
Total Posts:  347
Joined:  2008-11-14
Spain
 

There’s the last version. It works fine with 1.3.2.2.

function refresh_cache() 
    {    
        $this -> notify( 'Refreshing cache' );
        try {
            Mage :: getSingleton( 'catalog/url' ) -> refreshRewrites();
            $this -> notify( 'Catalog Rewrites was refreshed successfully', 'blank');
            } 
        catch ( Exception $e ) {
            $this -> notify( $e -> getMessage(), 'warning' );
            }
        try {
            Mage :: getSingleton( 'catalog/index' ) -> rebuild();
            $this -> notify( 'Catalog Index was rebuilt successfully', 'blank');
            } 
        catch ( Exception $e ) {
            $this -> notify( $e -> getMessage(), 'warning' );
            }
        try {
            $flag = Mage :: getModel( 'catalogindex/catalog_index_flag' ) -> loadSelf();
            if ( $flag -> getState() == Mage_CatalogIndex_Model_Catalog_Index_Flag :: STATE_RUNNING ) {
                $kill = Mage :: getModel( 'catalogindex/catalog_index_kill_flag' ) -> loadSelf();
                $kill -> setFlagData( $flag -> getFlagData() ) -> save();
                } 
            $flag -> setState( Mage_CatalogIndex_Model_Catalog_Index_Flag :: STATE_QUEUED ) -> save();
            Mage :: getSingleton( 'catalogindex/indexer' ) -> plainReindex();
            $this -> notify( 'Layered Navigation Indices was refreshed successfully', 'blank');
            } 
        catch ( Exception $e ) {
            $this -> notify( $e -> getMessage(), 'warning' );
            }
        try {
            Mage :: getModel( 'catalog/product_image' ) -> clearCache();
            $this -> notify( 'Image cache was cleared successfully', 'blank');
            } 
        catch ( Exception $e ) {
            $this -> notify( $e -> getMessage(), 'warning' );
            }
        try {
            Mage :: getSingleton( 'catalogsearch/fulltext' ) -> rebuildIndex();
            $this -> notify( 'Search Index was rebuilded successfully', 'blank');
            } 
        catch ( Exception $e ) {
            $this -> notify( $e -> getMessage(), 'warning' );
            }
        try {
            Mage :: getSingleton( 'cataloginventory/stock_status' ) -> rebuild();
            $this -> notify( 'CatalogInventory Stock Status was rebuilded successfully', 'blank');
            } 
        catch ( Exception $e ) {
            $this -> notify( $e -> getMessage(), 'warning' );
            }
        try {
            Mage :: getResourceModel( 'catalog/category_flat' ) -> rebuild();
            $this -> notify( 'Flat Catalog Category was rebuilt successfully', 'blank');
            } 
        catch ( Exception $e ) {
            $this -> notify( $e -> getMessage(), 'warning' );
            }
        try {
            Mage :: getResourceModel( 'catalog/product_flat_indexer' ) -> rebuild();
            $this -> notify( 'Flat Catalog Product was rebuilt successfully', 'blank');
            } 
        catch ( Exception $e ) {
            $this -> notify( $e -> getMessage(), 'warning' );
            }
        }

Regards!

 Signature 

amartinez {web/developer/designer} free modules: CustomImportExport Like it? Donate!

 
 
Posted: July 15 2009
   |  top  |  # 8 
 
Magento Community Magento Community
Magento Community
Magento Community
 
ho_nl
Sr. Member
 
Avatar
Total Posts:  118
Joined:  2008-06-25
Roelofarendsveen
 

Hi,

I cant seem to get it to work. Could you please elaborate how to implement your code. Just pasting your code in a file and running that obviously doesn’t work.

I’ve created a file that gets called by the crontab, but it throws an error, could you please give me an example of a correct file?

Best Regards,

Paul Hachmang

 Signature 

H&O;is specialist in eCommerce, webshops en in het bijzonder Magento, gevestigd in Roelofarendsveen (vlak bij Amsterdam). Wij bieden Magento consultancy, grafisch ontwerp, frontend ontwikkeling, technisch beheer en implementatie, maatwerk modules en performance optimalisatie. Opdracht uitbesteden? 071 744 0084. Blijf op de hoogte via onze website of volg ons op twitter.

 
 
Posted: July 15 2009
   |  top  |  # 9 
 
Magento Community Magento Community
Magento Community
Magento Community
 
amartinez
Guru
 
Avatar
Total Posts:  347
Joined:  2008-11-14
Spain
 

Hi!

You must “paste” this function wherever you want:

- in cron_refresh_cache.php (previously uploaded to this wiki http://www.magentocommerce.com/wiki/how_to_add_products_using_the_import/how_to_automatically_import_simple_grouped_and_configurable_products)

- in Productwith......php (refer to same wiki)

etc

 Signature 

amartinez {web/developer/designer} free modules: CustomImportExport Like it? Donate!

 
 
Posted: July 16 2009
   |  top  |  # 10 
 
Magento Community Magento Community
Magento Community
Magento Community
 
ho_nl
Sr. Member
 
Avatar
Total Posts:  118
Joined:  2008-06-25
Roelofarendsveen
 

Amartinez, Thanks for your reply.

The thing is that we are importing from a DBF (dbase file) file but that the layered navigation doesn’t function after the import. We know that running the Layered Navigation Indices does work (at least, when you run it, it does do something before it times out). Is it difficult to make the script as a stand-alone script that we can run using SSH?

 Signature 

H&O;is specialist in eCommerce, webshops en in het bijzonder Magento, gevestigd in Roelofarendsveen (vlak bij Amsterdam). Wij bieden Magento consultancy, grafisch ontwerp, frontend ontwikkeling, technisch beheer en implementatie, maatwerk modules en performance optimalisatie. Opdracht uitbesteden? 071 744 0084. Blijf op de hoogte via onze website of volg ons op twitter.

 
 
Posted: July 16 2009
   |  top  |  # 11 
 
Magento Community Magento Community
Magento Community
Magento Community
 
elawadi
Jr. Member
 
Total Posts:  6
Joined:  2009-05-04
 

I’m getting this error :
Fatal error: Uncaught exception ‘Mage_Core_Model_Store_Exception’ in /home/qbmfxrxo/public_html/shop/app/code/core/Mage/Core/Model/App.php:1267 Stack trace: #0 /home/qbmfxrxo/public_html/shop/app/code/core/Mage/Core/Model/App.php(633): Mage_Core_Model_App->throwStoreException() #1 /home/qbmfxrxo/public_html/shop/app/Mage.php(217): Mage_Core_Model_App->getStore(NULL) #2 /home/qbmfxrxo/public_html/shop/app/Mage.php(222): Mage::getStoreConfig(’web/url/use_sto...’, NULL) #3 /home/qbmfxrxo/public_html/shop/app/code/core/Mage/Core/Controller/Request/Http.php(189): Mage::getStoreConfigFlag(’web/url/use_sto...’) #4 /home/qbmfxrxo/public_html/shop/app/code/core/Mage/Core/Controller/Request/Http.php(141): Mage_Core_Controller_Request_Http->_canBeStoreCodeInUrl() #5 /home/qbmfxrxo/public_html/shop/app/code/core/Mage/Core/Model/App.php(293): Mage_Core_Controller_Request_Http->setPathInfo() #6 /home/qbmfxrxo/public_html/shop/app/Mage.php(434): Mage_Core_Model_App->init(’default’, ‘store’, Array) #7 /home/qbmfxrxo/public_html/shop/cach in /home/qbmfxrxo/public_html/shop/app/code/core/Mage/Core/Model/App.php on line 1267

on ver. 1.3.2.3 can any one help please]

 
 
Posted: August 15 2009
   |  top  |  # 12 
 
Magento Community Magento Community
Magento Community
Magento Community
 
xtrem57
Jr. Member
 
Total Posts:  6
Joined:  2009-03-12
 

Hi!

I’m getting the same error.

Can anybody help us getting this script to work?

Thanks

 
 
Posted: September 30 2009
   |  top  |  # 13 
 
Magento Community Magento Community
Magento Community
Magento Community
 
Martin
Guru
 
Avatar
Total Posts:  445
Joined:  2007-08-31
Brno | London | Los Angeles
 

Had the same issue. You have to change this snippet of code in the top of the script:

Mage::app('default');

to this:

Mage::app(Mage::app()->getStore());

 Signature 

Custom Magento Development
Extensions: Image Gallery | Category Hotspots | Order Comment Pro | more to come ...

 
 
Posted: November 23 2009
   |  top  |  # 14 
 
Magento Community Magento Community
Magento Community
Magento Community
 
leevo
Jr. Member
 
Total Posts:  22
Joined:  2009-03-21
 

Thank you for the great script, unfortunaly I can’t get is to work.

I keep getting the same error.

Fatal error: Uncaught exception 'Exception' with message 'Warning: session_start() [<a href='function.session-start'>function.session-start</a>]: Cannot send session cookie - headers already sent by (output started at /home/****/domains/****/public_html/tmp/cron_refresh_cache.php:2) in /home/****/domains/****/public_html/app/code/core/Mage/Core/Model/Session/Abstract/Varien.php on line 99' in /home/****/domains/****/public_html/app/code/core/Mage/Core/functions.php:239 Stack trace: #0 [internal function]: mageCoreErrorHandler(2, 'session_start()...', '/home/****...', 99, Array) #1 /home/****/domains/****/public_html/app/code/core/Mage/Core/Model/Session/Abstract/Varien.php(99): session_start() #2 /home/****/domains/****/public_html/app/code/core/Mage/Core/Model/Session/Abstract/Varien.php(154): Mage_Core_Model_Session_Abstract_Varien->start(NULL) #3 /home/****/domains/****/public_html/app/cod in /home/****/domains/****/public_html/app/code/core/Mage/Core/functions.php on line 239

I’m using the following cron_refresh_cache.php

(!$_SERVER["HTTP_USER_AGENT"]) or die ( "Nothing to do\n" ); // to run via local browser use ($_SERVER["SERVER_ADDR"] == $_SERVER["REMOTE_ADDR"])

require_once '../app/Mage.php';
umask(0);
Mage::app('');
$ver=Mage::getVersion();
$userModel=Mage::getModel('admin/user');
$userModel->setUserId(0);
Mage::getSingleton('admin/session')->setUser($userModel);

echo "Refreshing cache...\n";
Mage::app()->cleanCache();
$enable = array();
foreach (Mage::helper('core')->getCacheTypes() as $type => $label) {
    $enable[$type] = 1;
    } 
Mage::app()->saveUseCache($enable);

function refresh_cache() 
    {    
        $this -> notify( 'Refreshing cache' );
        try {
            Mage :: getSingleton( 'catalog/url' ) -> refreshRewrites();
            $this -> notify( 'Catalog Rewrites was refreshed successfully', 'blank');
            } 
        catch ( Exception $e ) {
            $this -> notify( $e -> getMessage(), 'warning' );
            }
        try {
            Mage :: getSingleton( 'catalog/index' ) -> rebuild();
            $this -> notify( 'Catalog Index was rebuilt successfully', 'blank');
            } 
        catch ( Exception $e ) {
            $this -> notify( $e -> getMessage(), 'warning' );
            }
        try {
            $flag = Mage :: getModel( 'catalogindex/catalog_index_flag' ) -> loadSelf();
            if ( $flag -> getState() == Mage_CatalogIndex_Model_Catalog_Index_Flag :: STATE_RUNNING ) {
                $kill = Mage :: getModel( 'catalogindex/catalog_index_kill_flag' ) -> loadSelf();
                $kill -> setFlagData( $flag -> getFlagData() ) -> save();
                } 
            $flag -> setState( Mage_CatalogIndex_Model_Catalog_Index_Flag :: STATE_QUEUED ) -> save();
            Mage :: getSingleton( 'catalogindex/indexer' ) -> plainReindex();
            $this -> notify( 'Layered Navigation Indices was refreshed successfully', 'blank');
            } 
        catch ( Exception $e ) {
            $this -> notify( $e -> getMessage(), 'warning' );
            }
        try {
            Mage :: getModel( 'catalog/product_image' ) -> clearCache();
            $this -> notify( 'Image cache was cleared successfully', 'blank');
            } 
        catch ( Exception $e ) {
            $this -> notify( $e -> getMessage(), 'warning' );
            }
        try {
            Mage :: getSingleton( 'catalogsearch/fulltext' ) -> rebuildIndex();
            $this -> notify( 'Search Index was rebuilded successfully', 'blank');
            } 
        catch ( Exception $e ) {
            $this -> notify( $e -> getMessage(), 'warning' );
            }
        try {
            Mage :: getSingleton( 'cataloginventory/stock_status' ) -> rebuild();
            $this -> notify( 'CatalogInventory Stock Status was rebuilded successfully', 'blank');
            } 
        catch ( Exception $e ) {
            $this -> notify( $e -> getMessage(), 'warning' );
            }
        try {
            Mage :: getResourceModel( 'catalog/category_flat' ) -> rebuild();
            $this -> notify( 'Flat Catalog Category was rebuilt successfully', 'blank');
            } 
        catch ( Exception $e ) {
            $this -> notify( $e -> getMessage(), 'warning' );
            }
        try {
            Mage :: getResourceModel( 'catalog/product_flat_indexer' ) -> rebuild();
            $this -> notify( 'Flat Catalog Product was rebuilt successfully', 'blank');
            } 
        catch ( Exception $e ) {
            $this -> notify( $e -> getMessage(), 'warning' );
            }
        } 
refresh_cache(); 
      
echo "Done\n";

?>

Could some one please help.

yours truly,

Harjo

 
 
Posted: December 8 2009
   |  top  |  # 15 
 
Magento Community Magento Community
Magento Community
Magento Community
Magento Community
Magento Community
    Back to top
Page 1 of 2
 
Search not working after import
Interfacing Magento & MS Dynamics Navision