Yeah, I build http://hnhiring.me well before the API via scraping. I looked into moving over after it was released but found it completely impractical for this sort of thing.

Hopefully it'll be improved at some point, especially given how bad the HN markup is (fun fact: all comments are at the same level in the dom; the indentation is achieved via a blank gif with a width property).

Ouch, that sounds painful. Glad I didn't go the scraping route :P

Firebase dev here, the websocket API would be much faster (less HTTP overhead and connection negotiation) and has the potential for more concurrent requests.

replace lots of these:

  return $http.get('https://hacker-news.firebaseio.com/v0/user/'+name+' + '/.json');

with

  return $q(function(resolve, reject){
    new Firebase('https://hackernews.firebaseio.com/v0/user/').child(name).on('value', function(snap) {
      resolve(snap.val());
    }, /*error handler here too*/)
  })


  https://www.firebase.com/blog/2014-10-07-hacker-news-api-is-firebase.html

  https://docs.angularjs.org/api/ng/service/$q

(yes the HN API design is a little funky though, but you can definitely improve the loading times with websockets)

Hey that IS awesome. I had no idea Firebase had a websocket API. Sweet!

send us a message at https://groups.google.com/forum/#!forum/firebase-talk if you have any problems.

The websocket API is the main selling point of Firebase, but I guess you were not actively looking to use Firebase in a product, rather, it was a necessity from HN API. Good luck! Don't hesitate to contact us.

Looking at the firebase api docs, I realized how unRESTful, their REST api is. No way to get a list of all stories. You can only get a list of ids and must then query the API with each id to get the content. If you plan on going this route, I def agree you store your results in your own backend db. If you want to stay stateless, look for a better HN job source, e.g., http://hnapp.com/?q=type%3Ajob+|+author%3Awhoishiring. Just one request and you can parse either the json or rss feed.

> Unfortunately, I need to make an HTTP request for every post (that's how the HN Firebase API works)

What's the total size of the jobs data? Would it be possible to grab all the JSON from the API, concatenate it and then self-host it as a single file? I imagine a $5 DO droplet with nginx would suffice for that.

It looks like firebase isn't even gzipping the JSON.

Yep certainly possible. It wasn't my aim thought as I wanted some Angular practice with promises :)