Which is the best football team in the country? No this isn’t an appeal for fans of Sunderland or Forest Green Rovers to pipe up and claim this title for their own. It is rather a request for an answer based on objective results, rather than personal preferences.
What if I were to answer the question by claiming that Stoke City is the best team in the country? No, I am not a fan of the club (I’m not even a football fan). I am simply using an objective measurement, which is that on the last day of the football season, Stoke City won 6-1 against Liverpool, which was the widest margin of victory by any club on that day in the highest league in the country. So Stoke are the best, and Liverpool are the worst. Right?
Now unless you are an avid fan of Stoke City, you are unlikely to agree with that claim. Even though objective data has been used — Stoke really did win 6-1 — nobody in their right mind would see this data — by itself — as nearly comprehensive enough or robust enough to substantiate the claim. Rather, most unbiased people would agree that if you want to know which team is the best in the country, the objective test is to look at which team attained the most points over the course of the season.
So the real answer is of course Chelsea, and the reason this claim can be justifiably made is that they amassed more points than any of their rivals (87 points over 38 games), and were therefore awarded the Premier League title. This doesn’t tell us whether they will be the best team next season — although it is highly likely given their performance this year that will again prove to be one of the best teams — but it does show that throughout this season, they performed consistently enough to earn the title of top team in England.
I mention this to illustrate a point that can be transferred over to the world of Labour Market Information (LMI). What is the purpose of LMI? It is basically to help us see the structural trends of a labour market, in order to help us understand what drives that economy. In other words, it is not merely a snapshot of what is happening right at the moment — the “Stoke” approach — but rather it is the use of data going back over a reasonable timeframe to give us a robust view of what that economy is really made of — the “Chelsea” approach.
One of solutions that takes the “Stoke” approach is what is often known as “real-time LMI”. Unlike “traditional LMI”, this does not rely on public datasets, but is rather derived from conducting a sweep of job vacancies and CV’s on the internet. There are of course benefits from this approach. Unlike “traditional” LMI, which relies on past data, a sweep of job postings and CVs can be done on a daily basis and therefore provides more up to date information.
However, although this approach can be useful, there are many reasons to treat it with a good deal of caution. Although job postings can help us understand aspects of a labour market, they cannot by themselves help us understand its structures and trends since they are only a momentary snapshot of that labour market. As we saw in the “Stoke” approach to football results, this way of creaming off real-time data can lead to some very misleading conclusions.
Imagine a region of the country associated with a certain type of manufacturing, and let’s say that one of the companies involved in that industry announces that they will be creating 50 jobs. This information might be useful, since it gives us a snapshot of the labour market in that region at that one particular time. However, it is of course a one-off and what it doesn’t tell us is anything of the structural trends that are driving that economy. For that, you are going to need data that goes back over a period of years.
In addition to this general problem, there are a number of other specific reasons why we should perhaps treat “real time LMI” with caution:
1. Employers do not always advertise their jobs
The number of actual jobs that are advertised is much less than the actual number of jobs out there. This is often especially true for highly skilled careers.
2. Job vacancies are often duplicated
This is especially likely when an employer goes through an agency as they often employ a variety of different methods and means of advertising a job.
3. A single job vacancy can refer to multiple jobs
Often a company will create a single online job posting, then select more than one employee from the applicants for that single job posting.
4. Not all jobs that are advertised will be picked up by real-time LMI sweeps
There is simply no way for every job on the internet to be picked up in a “real-time” sweep, and of course not all jobs are advertised on the internet.
5. Seasonal employment
Many jobs are seasonal, and so a snapshot of jobs postings at certain times – say in the run up to Christmas – will not give a true indication of labour market trends.
This is not to say that job postings are worthless. On the contrary, job postings can be extremely useful, for example when used in conjunction with “traditional LMI”, or perhaps for a college or university that is thinking of setting up short-term, customized industry training. However, by themselves, job postings or “real time LMI” cannot help us to understand the structural trends and demands of an economy, and they might very well lead us to making wrong conclusions and therefore to taking wrong decisions.
So the lessons are these: don’t confuse structural LMI with snapshot job postings. The two are not the same and the latter – taken on its own – may well lead you to “Stoke” rather than to “Chelsea”. And the second lesson is this: never make the mistake of confusing Chelsea with Stoke, or you may well upset Jose Mourinho!
For more information about the robustness of our data, contact Andy Durman at firstname.lastname@example.org