[PAID] Google Merchant Static Feed

[Martin]

Just released a very quick fix version 1.2.3-beta that includes:

• DropISO2 option added for single feed users
  (This drops the country code (eg GB) from the product id (eg: 25-GB -> 25 )
• brandExceptHide setting allows you to ALWAYS hide brand information for certain brands
• brandExceptShow setting allows you to ALWAYS show brand information for certain brands
  (Setting uses the brandID, not the label, so you will need to look up ID in table to use)
• FIX: Feed should complete properly now for very large product bases.
  (Small issue with the logic caused the tmp feed to be reinitialised every time it was run)
• (5.5.4 and earlier) Will get data from a custom field labelled "upc" (lowercase only!) and use as GTIN/UPC in feed (if data is available)

This is a quick fix and untested due to being away visiting family for the long weekend so test and report back any problems, and I'll resolve properly tomorrow/Wednesday.

Cheers,

Martin

[Martin]

The 1.2.3-beta was unfortunately flawed in the SQL syntax for pre-6.x versions of ISC and would produce a blank feed (they call these things beta for a reason).

1.2.4-RC3 has now been released which resolves this flaw.

Assuming all current bugs have been squashed I'll be making this a stable release and then moving on to developing the required functionality for coming Google changes.

[Martin]

Version 1.2.5 (Release Candidate #4) Released

Quite a few bugs squashed along with some functionality introduced along the way (No, it's not like Interspires "introduce more bugs" style... honest... *ahem*)

This release should resolve a number of issues caused by data display changes in 1.2.x version and in particular the problems that stopped large product feeds from being continued properly.


Changelog

• Version 1.2.5 RC#4 : Revision 307
  • 1.2.5 : Revision 307
  • FIX: Test for feed status failing due to file encoding incompatible with fgetc() php function
    (Now resolved with new test)
    .
  • FIX: Last product exported check failing for Incomplete feeds
  • FIX: Last product exported check failing when dropiso2 used for single feeds
    (Change in the ID field format now targetted with new Regex)
    .
  • Default $file_min_age_secs (minimum feed age) setting now 29 minutes
    .
  • Reformatted verbose output to provide clearer idea of success/failure
  • New verbose error/success reporting added to feed status checks
    (Makes troubleshooting much easier)
• Version 1.2.3 - 1.2.4 RC#3 : Revision 291
  • DropISO2 option added for single feed users
    (This drops the country code (eg GB) from the product id (eg: 25-GB -> 25 )
  • brandExceptHide setting allows you to ALWAYS hide brand information for certain brands
  • brandExceptShow setting allows you to ALWAYS show brand information for certain brands
    (Setting uses the brandID, not the label, so you will need to look up ID in table to use)
  • FIX: Feed should complete properly now for very large product bases.
    (Small issue with the logic caused the tmp feed to be reinitialised every time it was run)
  • (5.5.4 and earlier) Will get data from a custom field labelled "upc" (lowercase only!) and use as GTIN/UPC in feed (if data is available)

Code download/purchase available from the Snailsol BC store.
Note: Existing customers go to "Your Account" > "Completed Orders" to download the new version

[Martin]

Small last minute bug that's easy to fix:

Open: includes/classes/class.froogle.php

Find:

Code: Select all

$needle = "<\/feed\>";

Replace with:

Code: Select all

$needle = "(<\/feed\>)+";

This fixes the "undefined index: 1" error that appears when the script is run

[Martin]

Some news worth noting...


Enforcement of [Google Merchant] Unique Identifier Policy

As we announced on March 14, and in emails to all Product Search merchants, today we will begin requiring unique identifiers for products in all categories except for apparel, custom, and one-of-a-kind items. These requirements are part of an ongoing effort to make Product Search better for our users. Our unique identifier requirements enable us to build detailed product pages to help shoppers learn about products and compare prices from across the web. These unique identifiers are required for Google Product Search and Product Ads, and this policy will not impact Google Commerce Search.

We will be gradually enforcing this policy over the next few weeks, with full enforcement in effect on June 20th. During this time, products that do not have correct unique identifiers may not appear in our product listings. If you have products that do not have correct unique identifiers, we recommend that you make the necessary changes as soon as possible.

Please note that in the weeks after June 20th, we will begin disapproving the feeds of merchants who have a significant percentage of items that do not include the correct unique identifiers.

The current Google Merchant static module supports UPC/GTIN information for product listings but obviously you will need to have included this information in your product "Other" details to ensure you don't fall foul of the new requirement.

For those with ISC 5.5.4 or earlier, you can still add a custom field called "upc" (it must be lower case) with the appropriate UPC/GTIN/EAN value in it. This module will pick out that information and supply it as the UPC attribute in the feed.

No version of ISC currently supports the MPN properly, but instead relies on the product code (SKU). I have mirrored this in part but this module will:
- use the value for any custom field called "mpn" (lowercase required)
OR
- use the Product Code/SKU if custom field "mpn" doesn't exist and the product code field has a value



The other bit of news from Google is that Australia is now supported with its own targetted feed.
Google Shopping Arrives in Australia, Italy, Spain, and the Netherlands

The other countries listed have already had support implemented in this module but Australia can easily be added by including this line as follows:

Open: /includes/classes/class.froogle.php

Find:

Code: Select all

$this->country_streams['GB']	= 'GBP';	// United Kingdom,

Before, Add:

Code: Select all

$this->country_streams['AU']	= 'AUD';	// Australia

Worth reading that blog article (link above) to check the requirements but so long as you have a currency defined for Australian Dollars in your ISC setup you will find this module will support the feed.

That's about it for now... Still waiting for a bit more feedback from customers who have reported bug issues in previous versions, but at a guess the lack of additional feedback indicates the module is working as expected.

[Martin]

Version 1.2.6 (Release Candidate #5) Released

This release incorporates in the fix and new Australian feed posted above as well as a number of lingering issues with very large product bases.

I'd made a couple of logic mistakes that, coupled with an unusual timeout condition resulted in feeds being completed but without the file renaming required to make new feed available to Google (ie: foo.xml.tmp ->foo.xml). This was apparently due to PHP and/or Apache not killing the existing module run (when the process was overrunning timeout or similar) until it had closed the fopen() link to the tmp file. This stopped the module before it could publish the feed and resulted in tmp feeds just sitting, looking complete but not actually with the correct filename (ie: published). This issue has now, thankfully, been resolved and while the feed still times out, you simple need to re-run the module which now correctly recognises that the tmp feed file hasn't been published and will complete this process before moving onto any other country/target feeds remaining.

Another particularly annoying but potentially damaging issue was feed corruption in the form of duplicated products. This took place when the module was run in a browser and the page refreshed before the feed completed. This seemed to bypass all filelock tests and resulted in clone processes writing information to the same file and creating duplicates and a bad feed. A rather basic hack that checks the tmp feed filesize over a one second period, provided the solution and now the module can recognise if a zombie process is still writing the feed and just abort to avoid corruption.

Sorry if this all sounds very technical but this is for my own notes as much as anything else

Changelog

• Version 1.2.6 RC#5 : Revision 316
  • FIX: Resolves error "undefined index: 1"
  • ADDED: Australia target feed option
  • FIX: timeout issue with long feeds failing to publish/copy tmp feed to public feed location
  • FIX: bad logic test caused by tmp feed "completing" but not published.
  • FIX: "complete" tmp feeds will now publish on next run (as expected)
  • ADDED: debug setting allows you to limit product numbers to be exported to 500 max.
    (useful when trying to debug a VERY large product set)
  • ADDED: Logic test stops tmp feed corruption caused by browser refresh mid-feed generation

Code download/purchase available from the Snailsol BC store.
Note: Existing customers go to "Your Account" > "Completed Orders" to download the new version

[Mykatt]

Hi Martin,

Installed and using the Feed (and waiting for Google to let me know why it's not active), but found the following issues when setting up:

When running with Verbose mode on, I'm getting the following notice 8 times:

Code: Select all

Notice:  Undefined index:  124 in /.../public/www/includes/classes/class.froogle.php on line 1053

I found MPNs weren't being setup, so had to modify the following code removing the check for ShowProductSKU:

Code: Select all

				$mpn = '';
				if(key_exists('mpn', $row) && !empty($row['mpn'])) {
					$mpn = isc_html_escape($row['mpn']);
				}
				elseif($row['prodcode'] && $GLOBALS['ISC_CFG']['ShowProductSKU']) {
					$mpn = isc_html_escape($row['prodcode']);
				}
					
				if(!empty($mpn)) {
					$entry[] = sprintf("<g:mpn><![CDATA[%s]]></g:mpn>", isc_html_escape($mpn));
				}

Code: Select all

				$mpn = '';
				if(key_exists('mpn', $row) && !empty($row['mpn'])) {
					$mpn = isc_html_escape($row['mpn']);
				}
				elseif($row['prodcode']) {
					$mpn = isc_html_escape($row['prodcode']);
				}
					
				if(!empty($mpn)) {
					$entry[] = sprintf("<g:mpn><![CDATA[%s]]></g:mpn>", isc_html_escape($mpn));
				}

The Prodcode also seems to be the internal Product ID rather than the entered Product Code / SKU from the Product Information Page.

Otherwise, the feed seems to be working with the exception that every product is awaiting review.

[Martin]

When running with Verbose mode on, I'm getting the following notice 8 times:

Code: Select all

Notice:  Undefined index:  124 in /.../public/www/includes/classes/class.froogle.php on line 1053

Ok, that one you can ignore as just an annoyance.. Without being able to debug the code directly it seems either one of your categories has not been imported or deleted properly.

I found MPNs weren't being setup, so had to modify the following code removing the check for ShowProductSKU:

Ok... that'll be because you have the option to show product codes/SKU's disabled in your control panel. The "fix" is what you've done but it looks like it'll be worth providing options to override the product display options, so you don't have to do that anymore.

The Prodcode also seems to be the internal Product ID rather than the entered Product Code / SKU from the Product Information Page.

Ok, that's definitely odd...

Otherwise, the feed seems to be working with the exception that every product is awaiting review.

Are you based in Australia or just seeing the first page which shows the products for Australia first? I ask because every Australian feed is pending review... Mine has been for over a week now and Google's standard line is that they're checking everything.


To be 100% sure if you could email via info aT snailsolutions co uk with your ftp login details and an admin account for your store (limited to product editing and store log) I can debug the issue in situ for you.

[Mykatt]

The Prodcode also seems to be the internal Product ID rather than the entered Product Code / SKU from the Product Information Page.

Ok, that's definitely odd...

I'll have a trawl through the code to see if I can spot the issue here. Of course, I wouldn't be *that* fussed if Google Products allowed searching by product name rather than ID!

Otherwise, the feed seems to be working with the exception that every product is awaiting review.

Are you based in Australia or just seeing the first page which shows the products for Australia first? I ask because every Australian feed is pending review... Mine has been for over a week now and Google's standard line is that they're checking everything.

To be 100% sure if you could email via info aT snailsolutions co uk with your ftp login details and an admin account for your store (limited to product editing and store log) I can debug the issue in situ for you.

Yes - Australian feed. Got a reply back from Google that they're still evaluating every feed and there is a significant delay. Makes it hard to check what's wrong when they are very light on for details. We don't use UPC in most cases, so I've put a couple through but now need to see if they start showing before spending time updating all the products.

Fun and games really!

[Martin]

I'll have a trawl through the code to see if I can spot the issue here. Of course, I wouldn't be *that* fussed if Google Products allowed searching by product name rather than ID!

It's unlikely to be the code because I've tested the thing to death and it only grabs the productID for the <ID></ID>field.. Course I could have intro'd something recently too... but, as I said, if you can provide ftp access I can debug in situ.

Yes - Australian feed. Got a reply back from Google that they're still evaluating every feed and there is a significant delay. Makes it hard to check what's wrong when they are very light on for details. We don't use UPC in most cases, so I've put a couple through but now need to see if they start showing before spending time updating all the products.

If the feed is being generated properly there shouldn't be any problems with the information appearing... If you have a URL to the feed being generated I can check it's at least looking right...