Understanding microservices is a thing, actually developing one is another thing…
I don’t know how I should describe my feeling when I started working on the Auto Feed Discovery service which is part of Telescope’s microservices initiative. A bit excited, a bit nervous and maybe a bit confused. Despite having gone through two Udemy’s courses (20 hour long/each) on microservices, I am not that confident that I can develop one myself with minimal guidance. You know how in the course, the instructors normally hold your hand through out and you simply copy the code. I might understand it a bit more after the course but to write a service is actually a big challenge.
One good thing is that this one is not too complicated and at least I know where to start with the tips from professor Dave. Basically, I need to chain a few middlewares altogether before it can return an expected list of feed URLs. Some of them are very straightforward like checking whether the URL is valid or checking whether the URL is live. The most challenging part of this project is to discover the feed URL from the provided blog URL. My initial thought is to cut the corner and look for an npm package from whom have already done it. Surprisingly, there are not that many and the packages are not offering exactly what I need. After I tried out a few of them and failed, I decided to explore the code base of one package that might have some codes that I can use. And that was when I learn about cheerio, a fast, flexible & lean implementation of core jQuery designed specifically for the server. By looking at the logic of the code, I realized how the author discovers the feed URL from the blog link’s HTML by using cheerio to select the link element with type “application/rss+xml” and your feed URL will be in the href attribute. GREAT!!! Everything has become much easier now. I even thought about create a simple package just to discover feed since I don’t think there are a strong package to do that right now. It could be a future mini project to work on. I also think cheerio could be a powerful tool if I want to do web scraping for my ML/AI study and experience. I bet they have been using it already.
Dave has provided quite a list of feedbacks on my PR and I am planning to get on it sometimes this week (just noticed it’s already Thursday by the time I wrote this blog LOL). Well, it is what it is. When you are too busy, you lose track of time.
Thank you and until the next blog post!
Tony.
