We wrote here about how making unreasonable assumptions in the realm of Labour Market Information can ultimately be counterproductive. In this piece we want to identify another unreasonable assumption — one which can also lead to bad data and therefore bad decisions off the back of that data. This time the problem is that of taking data relating to general industries and occupations, and using it to make direct assumptions about trends within more specific industries and occupations.
As you may be aware, industries and occupations are classified by the government under 4-digit codes — SIC codes for industries and SOC codes for occupations. A 1-digit SIC or SOC code is the generic classification, representing a general industry or occupation, such as Manufacturing. A 4-digit code, on the other hand, is the more specific classification, representing a particular industry or occupation, for example the Manufacture of knitted and crocheted fabrics.
For Labour Market Information (LMI) forecasts to be accurate and useful, they must take into account trends at all digit levels. Sometimes LMI companies don’t do this, but instead take the high level data — say 2-digit industry or occupation data — and project the trends it shows straight onto the more specific 4-digit industries and occupations. So for example, they might take the growth rate for a 2-digit industry, and then assume that all 3 and 4-digit industries within this parent category are growing at this rate. This is a massive assumption — perhaps more a presumption — and it almost certainly leads to some very wrong conclusions.
Let’s take an example using some historic data. Looking at the 2-digit Health Professionals category for Hertfordshire, our data shows that between 2008 and 2013 there was growth in jobs of 12%. Now if we were to take this 12% figure and project it straight onto the more specific 4-digit occupations within the Health Professionals category, this would mean we would end up predicting 12% growth across all 15 4-digit occupations within the Health Professionals category. But is that what actually took place?
Actually no. Simple intuition alone tells us that such a uniform pattern is highly unlikely, but when we look at the actual data, we see that it is perhaps even more diverse than we might have suspected:
The red columns in the graph show the results we would have got had we projected the 12% change in the Health Professionals category across all 15 occupations within the sector. The blue bars, on the other hand, show what actually happened within these 15 occupations. You see the problem?
Now, we can make this a theoretical exercise and talk about these very different approaches all day long, but are there any practical ramifications of taking the first somewhat crude approach rather than the second more detailed method? Certainly.
Just do the following thought experiment: Imagine that back in 2008 you had had access to Labour Market Information that took the approach of mapping 2-digit SOC levels onto the more specific 4-digit categories. What would you have done? In the instance mentioned above, you would have predicted that both Veterinarians and Dental Practitioners would grow by 12% over the next five years. In actual fact, Veterinarians experienced massive growth of 66%, whilst Dental practitioners experienced a decline in jobs of 17%. Hence the answer to the question “What’s the difference between a Vet and a Dentist” is 83%! At least in Hertfordshire between 2008 and 2013 that is!
This is not ultimately a matter of data. Actually it is more a matter of occupations and skills. The fact is the 2-digit occupational classifications contain some wildly different occupations which often bear little or no relation to one another. For example, SOC code 34 is Culture Media and Sport Occupations, but down at the 4-digit level of this category we find such wildly diverse occupations as Artists, Photographers and Fitness Instructors. Would you be happy with data which assumes that because Culture Media and Sport Occupations as a whole are predicted to rise by, say 5% , we should assume that Artists, Photographers and Fitness Instructors will all rise by 5%?
More to the point, would you want to base your curriculum planning and apprenticeship programmes on this type of assumption? It is not that such data will always be wrong. As shown in the graph above, some of the predictions will be reasonably close. Yet such results are more a case of “Broken Clock Syndrome” — even a broken clock tells the right time twice a day — than a case of reasonable predictions based on good methodology.
Of course the real upshot of getting things this wrong is that it can end up costing money, jobs and credibility. Money, because a college basing its curriculum on wildly inaccurate predictions could end up making some really bad investments. Jobs, because training people in occupations that are actually declining may well reduce their chances of getting sustainable employment. Credibility, because a college that is training more and more people for occupations that are declining is hardly going to enhance its reputation in the eyes of employers or the public.
This is why we employ a team of expert data analysts and economists to ensure not only that our data is the most accurate and specific out there, but also that our projections, right down to the 4-digit level, are not arrived at by mapping high-level industries and occupations straight onto low-level industries and occupations, but are rather the result of sound economic methodology at all data levels. Were we not to go through this time-consuming process, we could have no confidence that we were giving you good forecasts, and you could have no confidence that your decisions were being made using the best intelligence.
For more information about our approach and methodology, email Andy Durman on firstname.lastname@example.org