Unfurling Product Information from Links

Introduction

We are very excited to announce that our OpenGraph unfurling API can now pick out product details and prices from URLs. As you can see below, our API produces easily consumable JSON with product information so you don’t have to worry about it!

Getting price information off of a webpage seems a fairly trivial task at first but shortly into attempting to implement something like this you will run into issues.

Handling the Popular Standards

Ok the good news is that standard formats have emerged to handle parsing site HTML but of course there are competing formats. The first format: Microdata - https://en.wikipedia.org/wiki/Microdata_(HTML) And then: RDFa - https://rdfa.info/ Then: JSON-LD - https://json-ld.org/ And finally: Good old fashioned meta tags - https://developer.mozilla.org/en-US/docs/Web/HTML/Element/meta All of the standard formats above are fantastic but support for them are fairly well spread out and there is of course a long tail of sites that adopt none of these.

The Long Tail

Just as is the case with the OpenGraph standard, the majority of sites out there tend to not follow any standards. Finding product images, descriptions, and prices on these sites require a very strong rules based parsing engine which is difficult to implement and an endless endeavor to maintain.

Bringing it All Together

Once you have dealt with parsing the standards and the long non-conforming tail, now you have to come up with yet another format to represent the data in a way that is predictable for your application.

Conclusion

If you are looking to implement something like this then please have a look at our API service. We perform all of these tasks and are continuously improving it.