Hi Thomas, really sorry you had a bad experience with ScrapingBee.
Would you mind sending me the account you used as I wasn't able to find anything under Thomas Isaac or Tillypa and couldn't see what was going wrong then.
I'm sure your comment has nothing to do with the fact that you share the same investor as ScraperAPI but I just wanted be sure.
If B4nan is around here from Apify, amazing work. On Crawlee, and on MikroORM. I especially use MikroORM extensively in production. One of the best, if not the best, ORM for NodeJS.
I wonder if they are including in that the money they already invested in the Ahrefs SEO tools. It seems to me that their SEO tools were already crawling a lot of content so this may be an extension of that / a new way to monetize the data. I personally know a lot of people who pay for their SEO tools so I imagine they have a decent amount of revenue.
There is a lot of intersection between Ahrefs tools and search. We were collecting web data for Ahrefs tools for 12 years, now we repurpose these data for Yep.
downside: doesn't seem to care much for boolean ("-keyword") search terms, instead opting for the modern thing of "let's actually search for that keyword, too!" v______v
We've decided to go this route because based on billions of web-scraped page, Headless-based scraping is still a minority. And, it's way harder and more expensive to do at scale.
PS: we spend tens of hours writing those piece of content and even pay a technical editor to spot the typo and make it more readable since we're not native English. You might not like this post, but I can assure that genuine care was put into writing this!
Scrolling through your article I disagree, it's high quality content. What converts it to "low quality" is the bait-n-switch title. This is not "everything you need to know" -- this is "how to get started from scratch".
Metaphor would be "Everything you need to know about fixing cars" and the article shows you how to check the engine light, change oil, rotate tires, and replace spark plugs. There's just no way to make a promise that large and have your article be considered high quality.
I thought it was a nice summary, concise, organized, with examples and references. Will revisit it should I need a reminder on scraping. Would not call it low quality at all.
Would recommend you ignore passing comments with no constructive criticism. The title is going to be a point of contention as it’s a big claim and probably being misinterpreted as not “everything you need to know [to get started]” but rather “everything you need to know [ever is in this one article and you’ll need not read anything else]”.
I don’t think it’s very good and many other highly-rated top level comments seem to agree that not only does it have a scammy SEO “top ten best ${X} in ${CURRENT_YEAR}” but there is a mismatch between what the article is attempting to do with how it is attempting to explain and do it.
While I’m glad it’s not GPT-3 level spam, or outsource to third world country for copy level spam, in my opinion the article fails in several fundamental ways, noted above. Putting “genuine care” into something is commendable, but is not a substitute for quality, relevant content.
OTOH you’re getting lots of clicks and views for whatever product you’re selling, and even my comments help the “traction” HN gives it, so it doesn’t actually matter what I think.
I think it is good. You have to remember that some of the loudest voices on here take things extremely literally. They have no concept of hyperbole for emphasis. Or, indeed, anything which makes writing interesting to read.
Is your guide everything someone needs to know? No. But anyone literate in the ways of modern English understands what you mean.
It is an excellent guide and I think you should consider expanding it & perhaps creating a book.
Please don't be discouraged by the people on here who don't have the skill or courage to write or submit anything.
While it's true I often end up needing something like selenium, it's way more heavy handed and I usually reach for it last. It doesn't scale as well, harder to troubleshoot IMHO, and more libraries and dependencies to deal with in a language where that's already not great.
The problem with brinksmanship is that you have to be prepared to go through with the act you threaten and accept the consequences of it. Take Snake Island for example where the response was literally "fuck you". There were only 13 people defending the island yet Russia spent an enormous amount of money to level the place by naval bombardment rather than attacking by landing troops. Because ultimate the threat isn't hollow and the island itself is pretty worthless.
With threats of nuclear weapons it:
- Makes Putin and Russia look weak, they have to threaten literally the worst weapons they have to beat Ukraine.
- Actually using them might cause nuclear retaliation.
- Actually using them is going to absolutely cause an enormous international response and almost certainly remove any allies they have in the US etc.
- Actually using them is going to ruin a portion of the country you want to take owner.
"We do not encroach on others, but we will not give up our own. We have a wonderful army. Our guys have a unique combat experience and modern weapons.
This is an army many times stronger than eight years ago. We are told that February 16 will be the day of the attack.
We will make it the Day of Unity. The relevant decree has already been signed. On this day, we will hoist national flags, put on blue and yellow ribbons and show the world our unity.
We all want to live happily, and happiness loves the strong. We have never been able to give up and we are not going to learn that, ”
So, the great army WAS TOLD when the attack would be, i.e. relies on the interested party's intelligence. That's not a strong army, that's a puppet mercenary.
A strong army won't trust what a demented president said especially when none of his people confirmed that. Maybe he confused the topic of the discussion - nothing new for Sleepy Joe!
Would you mind sending me the account you used as I wasn't able to find anything under Thomas Isaac or Tillypa and couldn't see what was going wrong then.
I'm sure your comment has nothing to do with the fact that you share the same investor as ScraperAPI but I just wanted be sure.