Comments on: “Vanity” or “Specific value” benchmarking?

By: Information Security Governance and Benchmarking - Topic Research, Trends and Surveys

Sun, 18 Jul 2010 00:23:43 +0000

[...] bright: Infonetics Research recorded growth of 70 … market research, surveys and trends NetJMC / Intranet Strategies Global Organizations / "Vanity" or … I think it is a vanity exercise to participate in group benchmarking exercises for intranets. I [...]

By: Chieftech

Chieftech — Mon, 25 Jan 2010 11:25:29 +0000

Some interesting comments. I wouldn’t say that intranet benchmarking is a vanity exercise, quite the opposite. However, as with all types of benchmarking, there is a risk that rather than changing perceptions of poor performance it may instead reinforce a particular world view. I see this as a particular risk with intranet benchmarking because of the economics of running such a benchmarking exercise. I’d add to Mark’s comment that “Benchmarking only has value if it emerges into actions, decisions and behaviours” that support your objectives. In other words, its all relative.

By: Andrew Wright

Andrew Wright — Sun, 24 Jan 2010 13:27:54 +0000

Thanks for clarifying Jane. I’ve found this topic to be very interesting and informative.
I agree with you 100% that to demonstrate the value of their intranets, organisations should be putting more time and effort into identifying indicators specific to their unique business. As I said in my original post, benchmarking (at least in the case of the WIC) should be just one of a number of tools used to improve intranets – not an end in itself.
Keep up the great work with the Intranet Trends Report – every manager I know who has the report has found it to be a very useful resource.

By: Jane McConnell

Jane McConnell — Fri, 22 Jan 2010 15:15:40 +0000

First I’d like to make it very clear that my original post was not at all directed to organizations such as IBF or the WIC who conduct benchmark activities. It was written as an expression of the frustration I feel when I see so many enterprises who are not able or even attempting to define their own, specific business indicators to help demonstrate the value of their intranet and how it is/has/or could contribute to the core mission of their organization. I’ll write a post with some figures about that soon.
I’m a long time supporter of IBF, starting way back to when, as Luke, there was no global IBF, no social media activities and no Intranets Live which, by the way, I consider a major advance in the intranet-sphere. I wrote a post after the first IBF24 in June 2008.
I also blogged about IFVB last April Intranet Financial Value Benchmarking in April 2009.
As for the Worldwide Intranet Challenge, I have seen it in use firsthand with one of my clients, and find Andrew’s end-user initiative very useful and thought-provoking. I recently recommended it to the over 400 members of NetJMC&Co (Linkedin group for intranet managers.)
Mark:
I agree completely that “good benchmarking, provides us not only with the ‘snapshot’ but the roadmap for the continuing journey.”
Luke:
You’re right! This has snowballed into a great conversation. I’m going to do a second post summarizing the key points in all these comments, so that someone just coming upon this discussion can see the highlights.
If possible, it would be very useful to hear back from you later when you have developed your “financial metric or ROI that shows that the work we are doing is having a good effect”. In my experience, and that was part of my initial post, that is the real key to what intranet managers are trying to do.
I’m glad you used an “or” when you said “financial metric” OR “ROI” because many people fall into the trap of thinking ROI = $, and forget that it means “return on investment” of different sorts.
Andrew:
Your first question: “…how do you reconcile these differences when you assess the stages of maturity of an intranet, as you do in the Intranet Trends report?”
I do not attempt to reconcile differences at this level, because the survey is much more macro. It does not identify specific goals of specific enterprises, then make comparisons among those with similar goals.
When the survey population grows larger (we’re just reaching 300 organizations as of 2009), then some sectorial analysis could be valid. For the moment, I am beginning to be able to make general observations based on the primary type of workforce in the organizations: knowledge-based, rules-based or manufacturing-based.
Your second question: “Doesn’t this [classifying intranets by level of maturiy] imply that intranets, regardless of size or sector, have similar needs and can therefore be measured and compared?
I defined the 3 stages of maturity once I was into the 3rd year of the survey. (Next year will be the 5th year). I needed a broad framework that would provide structure and meaning to the many observations and trends coming out of the research. The 3 stages are described with general attributes not specific indicators.
I do not score the survey organizations individually, nor rank them. I don’t develop action plans in the context of this research. Also, in very large enterprises, there is always a mixture of stages, i.e. pockets throughout the intranet landscape with varying degrees of maturity.
The purpose of the research I conduct through this survey is to observe what is happening and to identify trends. Thus the title “Global Intranet Trends” By seeing what enterprises are doing, looking at what they are planning to do, and by seeing the differences depending on the 3 stages, I can spot things that are:
- starting in most intranets regardless of maturity
- just starting in the more mature intranets
- causing difficulties for intranets
- and so on.
Again, this is very macro.
From the intranet manager feedback I get, I know that many are using the report itself and the yearly questionnaire (of which around 50 % of the 120 or so questions evolves year to year) as a reference point for their own thinking and internal benchmarking. In these cases, I’d say my framework provides guidance for their thinking.
What you, IBF and others provide are actual benchmarking services.

By: Jane McConnell

Jane McConnell — Thu, 21 Jan 2010 06:46:56 +0000

What a conversation we are having! I’m offline most of today because of travel and will respond tomorrow.

By: Andrew Wright

Andrew Wright — Thu, 21 Jan 2010 05:27:09 +0000

Jane – I am interested in your comments about working with large organisations & sectors and the different requirements they have.
You mention that they have ‘specific needs so different when it comes to what they are trying to achieve with the intranet that I doubt comparing themselves to others provides real answers’. Then you also mention ‘I have seen how intranet practices vary and should vary in order to suit the enterprise, the people and the business they are in.’
My question is how do you reconcile these differences when you assess the stages of maturity of an intranet, as you do in the Intranet Trends report? To me, categorising intranets by level of maturity is a type of benchmarking (a very helpful categorisation I should add). You are comparing intranets with other intranets. Doesn’t this imply that intranets, regardless of size or sector, have similar needs and can therefore be measured and compared? Or have I misunderstood something?

By: Luke Oatham

Luke Oatham — Wed, 20 Jan 2010 23:36:20 +0000

Gosh Jane what did you start?
I’m a user experience contractor and especially involved with public sector, governmental bodies over the past 10 years.
A handful of companies that I’ve worked with along my travels have been IBF members and have been benchmarked once or more. In my experience, it is these companies that have had intranet teams that are very “aware” of their intranets; very on the ball about usability and accessibility issues, great with search and content, and have some governance and strategy in place.
Where I am now (95,000 staff), we’ve just launched our new intranet with a heap of usability improvements, many which have surfaced through benchmark reviews. For 2010 we need to address areas such as employee engagement, interactivity and staff directory on the intranet. We know where we need to go next, but it’s good to go find out what others have done/are doing and how we can use this knowledge and bend it to our own needs. And of course it’s good to hear about other people’s mistakes so we can avoid them!
Working in the public sector I am hoping that our scores go up when we are next benchmarked as I know that the improvements that we’ve recently made will help improve staff efficiency and ultimately save public money. Of course, it’s good to be able to show our bosses the changes that we’ve made and the results.
I think that the IBF has a very good feel for the lifecycles and evolution of intranets, across a wide range of companies. And it’s good to pick and choose which bits of that apply (sticking with your own sector). I know that where I’m working now, I’m not interested in addressing multi-language or global workforce issues. So I don’t take part in these discussions/events. We’ve never been benchmarked on governance since we just aren’t there yet.
And IBF has changed with the times over the years. I can remember when there was no member extranet, no social media interaction and no global contacts.
I’m really excited about the year ahead and hope to get my teeth into more intranet development projects. For me, number one priority is the staff directory. And I’m also very interested in trying to develop some financial metric or ROI that shows that the work we are doing is having a good effect.

By: mark Tilbury

mark Tilbury — Wed, 20 Jan 2010 23:11:03 +0000

All very logical comments regarding the information enterprise as a whole, viewing the wider picture etc. Having worked both inside and outside the firewall (i.e. website, intranets, extranets) for twelve years, the motivation from senior management hasn’t really changed. They want to know what the competitors are doing, how do we compare, how can we do better. It’s also a sad fact that many companies turn a deaf ear to employees yet listen with great attention to the same message from an external representatives.
For public websites we looked over the garden fence, copied, imitated or made judgements. In other areas not as visual as websites, we have spent fortunes in attracting staff to replicate what competitors had.
For intranet’s it hard to look over the fence.That’s where the value of benchmarking comes in. You could argue it stifles innovation or does little to develop further the growth of the intranet. I would disagree. As Jane mentions, it answers questions from senior managers, and in many cases it brings ‘big players’ to the company table. Measuring, benchmarking or copy organisations can lead to cloning (a particular fault in all professional services industries), however, it does bring security and validation. Organisations are not logical. Whether it’s driven by egos or politics (or both) a benchmarking report certainly becomes an asset in dealing with internal personalities. That alone could justify benchmarking for me. But there is more.
Benchmarking only has value if it emerges into actions, decisions and behaviours. Acquiring a ‘benchmarking artefact’ is only the first stage of what’s interesting about the process. IBF have conducted a number of benchmarking reports within my organisation. Not only did they provide the expected guidance on lifting us up to industry standard, they were also guiding us on potential actions, decisions and behaviours. I would suggest benchmarking, or good benchmarking, provides us not only with the ‘snapshot’ but the roadmap for the continuing journey.

By: Jane McConnell

Jane McConnell — Wed, 20 Jan 2010 22:09:18 +0000

Hello Nancy, Andrew and Martin,
A global thanks to you all for your detailed responses.
Andrew:
What prompted my post is in fact several years of hesitation I’ve had when I hear about benchmarking methods. As you know, my own clientele are the very large organizations, usually over 20-30,000 employees.
I find their specific needs so different when it comes to what they are trying to achieve with their intranets that I doubt that comparing themselves to others provides real answers. Each enterprise is at a particular stage of development, with its own culture, its strong points and weak points, its business goals and its overall values – all of which evolve.
I find that identifying their own indicators is more meaningful. That’s how they can see and prove the value the intranet brings.
That said, as you know, I saw the results of the WIC one of my clients did, and it was extremely useful for specific work they and I are currently doing. However, the main purpose it served was as a wake-up call regarding certain aspects of the intranet. It did not provide concrete leads as to what needed to be done, although it certainly indicated the weak and strong areas, generically speaking. As a wake-up call, it was invaluable!!
Nancy:
One definition (Wikipedia) is “the process used in management in which organizations evaluate various aspects of their processes in relation to the best practice, usually within their own sector.” It is the last 4 words “within their own sector” that are key for me.
Having worked with many companies in different sectors over years, I have seen how intranet practices vary and should vary in order to suit the enterprise, the people and the business they are in.
Martin:
You put your finger on it when you say “Benchmarks tell you exactly where you are, and you need to know that before deciding how to get to where you need to go.”. I agree that the first step is to know where you need to go.
However, I don’t believe organizations need to know what the future holds in order to know where they “need to go”. This should be part of their strategic planning, done with different scenarios and options.
We should start doing intranet strategy plans based on different future scenarios of the enterprise.
Let’s continue this conversation, everyone.

By: Jane McConnell

Jane McConnell — Wed, 20 Jan 2010 21:49:27 +0000

Hello Luke,
It’s good to hear about the experience from an “end-user” viewpoint, I mean an enterprise who’s done benchmarking, not the groups or consultants who offer it.
I see the advantages for you from your detailed explanations.
I’m curious to know how many years you have been a member of IBF and how many benchmarks you’ve done?
Thank you very much for sharing. Your firsthand views are exactly what I am trying to understand!
Jane