TIOBE index is flawed
For the first time, Kotlin has made it to the top 20 of the TIOBE index. One can boldly ask - why so late? After all, Kotlin is one of the most popular programming languages. The problem is not Kotlin or its users, the problem is the TIOBE index itself.
And in this article we are going to explain why you should stop using TIOBE index to gauge programming language popularity.
Is it Kotlin or is TIOBE?
Kotlin has been gaining popularity for several decent years now. Its biggest leap happened when Google chose it as the official language for native Android apps. It also gains more and more recognition on the backend, simplifying the development process, thanks to a more expressive syntax.
We've looked at the data in our IT Community Survey 2024, concerning knowledge of programming languages, Kotlin took 10th place - 7.1% of professional developers declared they use it at work regularly. In the StackOverflow Survey 2023, 9.7% of developers declared knowledge of Kotlin. This gave it the 15th position.
In the TIOBE ranking, Kotlin has made it to the top 20 for the first time in 2023. Okay, maybe it would be nothing unusual if it weren't for the languages that rank higher than Kotlin.
This elite group of languages, that are placed higher than Kotlin, includes:
- Visual Basic (7th place)
- Fortran (11th place)
- Matlab
- Scratch
- Delphi/Object Pascal
Something’s fishy here.
None of these languages even appear near Kotlin, neither in our survey focused on the situation in Poland nor in the StackOverflow survey, nor any other industry survey or ranking.
It's strange because TIOBE claims:
The TIOBE Programming Community Index is an indicator of the popularity of programming languages. The index is updated once a month. The ratings are based on the number of qualified engineers worldwide, courses, and third-party vendors.
If so, it's weird that JavaScript is only in 6th place (right before Visual Basic), C and C++ occupy 2nd and 3rd place, and Java and C# are behind them. Ever heard of TypeScript? That's good, because TIOBE did not.
The Problem with TIOBE
What appears in the index results directly stems from its methodology, which is presented on the TIOBE website:
Basically the calculation comes down to counting hits for the search query
+"<language> programming"
It seems to us that the number of results for such a query may not translate into the actual popularity of programming languages.
This index definition will reward languages taught at schools or universities - hence the presence of Matlab or Scratch in the index, or even Python's first place in the index.
Languages that do not have a very developed ecosystem will also be promoted. E.g. React results won't be counted as JavaScript-related. Besides, who the heck writes "JavaScript programming" anyway?
In TIOBE's methodology many online retailers are counted as "search engines". This basically counts books available on the topic.
Now you know why we are not fans of the TIOBE index. And let's be clear - it's not about hating the TIOBE as a company - although when they write that their "ratings are based on the number of qualified engineers worldwide", it doesn't look good for them.
However, that's not the main point. We are on the internet, and not everything that glitters is gold. Before using data from any index, it's important to check the methodology behind it. This is especially important because the results of various popularity indexes are often talked about in many online publications. This responsibility should fall on the outlets, not on you, but that's the reality.
What to use instead of TIOBE?
When we started Bulldogjob a few years ago, there wasn't much data available on tech, especially on programming. Times have changed, and now we have much more information about our industry.
There are a few noteworthy surveys; here are some examples:
- Stack Overflow Survey
- State of Developer Ecosystem (by Jetbrains)
- State of JS (and other frontend technologies)
- Polish IT Community Report (by us, shameless plug 👀)
Also there are surveys specific for smaller ecosystems, like Go Developer survey. You'll often found similar sources for other languages.
If you are looking for results based on what's going on in the interwebs, then consider RedMonk ranking, that analyze language popularity through GitHub and Stack Overflow statistics.
We don't claim any of these are perfect (including our own survey). Each of the mentioned sources has its own strengths and weaknesses, so do not treat them as an objective source of truth. Just do your research.