Repository navigation
receiving contacts/social media accounts for a given url #2
Description
I built quick version, and not yet implemented the list
- Extract data from parsed structured data.
e.g.
<script type="application/ld+json">
{
"@context": "http://schema.org",
"@type": "Organization",
"name": "Let's Validate",
"url": "https://www.letsvalidate.com/",
"logo": "https://www.letsvalidate.com/img/logo.png",
"email": "info@letsvalidate.com",
"description": "Site launch checklist checker",
"sameAs": [
"https://www.facebook.com/letsvalidate",
"https://m.me/letsvalidate",
"https://twitter.com/letsvalidate",
"https://plus.google.com/+letsvalidate",
"https://vk.com/letsvalidate"
]
}
</script>- Extract data from parsed meta data of twitter card.
e.g.
<meta name="twitter:site" content="@letsvalidate">- Prerender web apps before extracting data.
- Deeply crawl, e.g. contact page
- Maybe to use Google knowledge graph too?
the endpoint:
https://api.letsvalidate.com/v1/contacts?url=docker.com&prettify=true
result:
{
"url": "https://www.docker.com/",
"originalUrl": "http://docker.com",
"contacts": {
"email": null,
"fax": null,
"tel": null,
"socials": [
{
"domain": "twitter.com",
"id": null,
"name": "docker",
"confidence": 100,
"url": "http://twitter.com/docker"
},
{
"domain": "youtube.com",
"id": null,
"name": "dockerrun",
"confidence": 100,
"url": "http://www.youtube.com/user/dockerrun"
},
{
"domain": "facebook.com",
"id": null,
"name": "docker.run",
"confidence": 100,
"url": "https://www.facebook.com/docker.run"
}
]
}
}@JHabdas What do you think, is it worth to implement it or already available such api?
Activity
hbakhtiyor commentedon May 30, 2017
wow, thanks a lot for your advices and taking your times.
using headless chrome only for capturing screenshots, for js rendering, i consider https://github.com/scrapinghub/splash for its lightweight
how about the idea itself? anyone will be interested in?