Thank you, that was educational! At the time I'd have been happy with just getting the data out, so to encourage others, here's a simpler version of the query: https://w.wiki/3x8t
Short version:
SELECT ?awardYearLabel ?winnerLabel ?dateOfBirthLabel WHERE {
SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
?statement ps:P166 wd:Q185667.
?winner p:P166 ?statement.
?statement pq:P585 ?awardYear.
?winner wdt:P569 ?dateOfBirth.
}
ORDER BY (?awardYearLabel)
Annotated version with comments:
SELECT ?awardYearLabel ?winnerLabel ?dateOfBirthLabel WHERE {
# Boilerplate: Provides, for every "?foo" variable, a corresponding "?fooLabel"
SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
# "Statements" of the form "<subject> <predicate> <object>."
# also known as "<item> <property> <value>."
# Variable names start with "?" and we can think of them as placeholders.
# For example, a straightforward query that lists winners
# ("P166" means <award received> and "Q185667" means <Turing Award>):
# ?winner wdt:P166 wd:Q185667. # <?winner> <received award> <Turing Award>
# "Qualifiers" on statements: See
# https://wdqs-tutorial.toolforge.org/index.php/simple-queries/qualifiers/statements-with-qualifiers/
# or https://en.wikibooks.org/wiki/SPARQL/WIKIDATA_Qualifiers,_References_and_Ranks
# A **statement** of the form "<somebody> <received award> <Turing Award>"
?statement ps:P166 wd:Q185667.
# In that statement, the <somebody> we shall call "?winner".
?winner p:P166 ?statement.
# That statement has <point in time> qualifier of "?awardYear".
# ("P585" means <point in time>)
?statement pq:P585 ?awardYear.
# The ?winner has a <date of birth> of ?dateOfBirth.
# ("P569" means <date of birth>)
?winner wdt:P569 ?dateOfBirth.
}
ORDER BY ?awardYearLabel
?awardYear and ?dateOfBirth are literals, so you don't need to take *Label of them (that's only useful for Qnnn nodes).
Below I use a blank node (since you don't need the URL of ?statement) to simplify the query, and calculate the age as a difference of the two years:
SELECT ?awardYear ?age ?winnerLabel WHERE {
SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
?winner p:P166 [ # award won
ps:P166 wd:Q185667; # Turing award
pq:P585 ?awardDate]; # point in time
wdt:P569 ?birthDate.
bind(year(?awardDate) as ?awardYear)
bind(?awardYear-year(?birthDate) as ?age)
}
ORDER BY ?age
I think this is an interesting case because scraping this is easy (just one page) where the wikidata query requires dealing with modifiers which is a bit more complex.
(It requires the birth dates, so it is more than one page)
The HTML structure may change over time: if the request is executed few times over a long period, the scrapper may/will require more maintenance than the SPARQL request.
A very common argument in HN comments that discuss the merits of so-called web APIs.
Fair balance:
Web APIs can change (e.g., v1 -> v2), they can be discontinued, their terms of use can change, quotas can be enforced, etc.
A public web page does not suffer from those drawbacks. Changes that require me to rewrite scripts are generally infrequent. What happens more often is websites that provide good data/information sources simply go offline.
There is nothing wrong with web APIs per se, I welcome them (I use the same custom HTTP generator and TCP/TLS clients for both), but the way "APIs" are presented, as some sort of "special privilege", requiring "sign up", an email address and often more personal information, maybe even payment, is for the user, cf. developer, inferior to a public webpage, IMHO. As a user, not a developer, HTTP pipelining works for me better than many web APIs. I can get large quantities of data/information in one or a small number of TCP connections (I never have to use use proxies nor do I ever get banned); it requires no disclosure of personal details and is not subject to arbitrary limits.
What's interesting about this Wikidata/Wikipedia case is that the term chosen was "user" not "developer". It appears we cannot assume that the only persons who will use this "API" are ones who intend to insert the retrieved data/information into some other webpage or "app" that probably contains advertising and/or tracking. It is for everyone, not just "developers".
The semantics of RDF identifiers drift at least as often as HTML format changes.
For example, at one point I was doing a similar thing against DBPedia (a sort-of predecessor to WikiData).
I was doing leaders of countries. But it turns out "leader" used to mean constitutional leadership roles, and at some point someone had decided this included US Supreme Court Chief Justice (as the leader of the judicial branch).
So I had to go and rewrite all my queries to avoid that. But most major countries had similar semantic drift, and it turned out easier to parse Wikipedia itself.
Disclaimer: I follow https://www.youtube.com/channel/UCp2i8QpLDnWge8wZGKizVVw / https://www.twitch.tv/belett (mostly in French, sometimes in English).
Without these courses, I wouldn't have been able to write this request.