Scenario Quality Ranking

From ScenarioThinking
Revision as of 14:08, 9 December 2004 by Carmel (talk | contribs)
Jump to navigation Jump to search

Group 1
<Rank High> Internet Commerce: free internet service provision 2003 (1996) 

Good contents: Clear explanation in Trend and Matrix makes Scenario convincing.
Good presentation: Easy to follow, nice visual impression

<Rank Middle> Health: health in 2010 (1996) 

OK contents: Interesting contents, but relationship between driving forces and scenarios is vague.
OK presentation: Easy to follow, but not so well visual impression


<Rank Middle> Work and Organizations: Organizations 2020 (1997) 

OK contenst: The story is credible, but sometimes too complex and little techonoly implications
OK presentation: The prentation is easy to follow, could include some visuals


<Rank Low> Industry sectors: Leisure 2010 (1999) 

OK contenst: The story is credible, but future can be a mixure of all scenarios
Not good presentation: Not easy to follow, not clear sequence of the project


<Rank Low> Telecommunications: telcoms 2003 (1996) 

Not good contents: Overall story is not convincing. The axis for scenarios seems unreflected.
OK presentation: Easy to follow, but not so well visual impression


Group 2
BEST Human Relationships in 2015

Reasons: Original, easy to understand, linkage between elements

Telecommunications in 2015

Reasons: Well-researched, detailed, in places uncany in predicting future. Signposts were good. Crowded waters scenario especially strong.

Leisure in 2010

Reasons: Well researched, organized and presented.

Distance Education in 2010

Reasons: Easy to read, nice framework, it goes to the point

Branding in 2005

Reasons: easy to understand, takes into account not only marketing concepts

Workspace in 2010

Reasons: some parts were really good, but others not. Overall, it doesn't look like an integrated document/proposal. No conclusions.

Food Retailing in 2006

Reasons: Focused on Alberthein in Netherlands too narrow, Difficult to read, Not very revolutionary ideas

Electronic Cash in 2010

Reasons: Disorganized and not so clearly presented, there is no clear explanation how electronic cash can affect society.

Telecommunications in 2003

Reasons: Not well thought out. Unclear. Basic assumptions left unstated. Research was very poor. Scenarios not believable--actors did things that were illogical given the scenario laid out.

Internet in 2005

Reasons: Unoriginal, not forward looking enough, no linkage between elements, badly organized


Group 3
Dear scenario-thinkers,

please explore our way of thinking and our results!

Short introduction to our evaluation process:

  1. Developing criteria catalogue for assessing scenarios: five process, three quality, three presentation criteria
  2. Screening all scenarios
  3. Chose 5 most recent ones (4 group scenarios, 1 class scenario) to analyze in depth
  4. Everyone of group individually assessed these 5 scenarios according to the criteria catalogue (reasoning: everyone has same starting point for discussion, increase common understanding, learning experience of group larger)
  5. Merging the individual results and discussing the final ranking


Criteria catalogue for assessing scenarios:

  • Process, quality and presentation (effectiveness, efficiency and presentation)
  • Weighting 40% - 40% - 20% to emphasize content over lay-out
  • Please also refer to the attached picture


Ranking results


Description

Rank 1: Genetic revolution

  • Process: Introduction is present, good/deep structured assessment of driving forces, indicators and monitoring process missing
  • Quality: very consistent in approach, detailed argumentation
  • Presentation: good structure and creative presentation of the scenarios


Rank 2: Leisure

  • Process: reflected on indicators and implications, chosen matrix easy to understand, causal relationship scheme missing
  • Quality: no referencing present, consistent in approach, broad mindset
  • Presentation: no logical structure, not attractive/boring coloring


Rank 3: Interpersonal communication

  • Process: no focal issues, good causal scheme but no interpretation, indicators and monitoring process missing
  • Quality: missing depth in scenarios, consistent however
  • Presentation: well structured, nice layout but cold be more entertaining, no consistency in the language


Rank 4: Childhood Freedom

  • Process: bad introduction, missing steps, over structured in depth of driving forces (image as a whole not present, confusing), choice of axes mysterious
  • Quality: reasoning and in-depth analysis lacking, consistency good, good wrapping of ideas
  • Presentation: original, logistics are hidden


Rank 5: Information Society (Class project)

  • Process: no introduction, no focal issues, no reasoning behind thinking, no transparency, no causality
  • Quality: no consistency, no sufficient depth of argumentation
  • Presentation: no structure, no consistency in lay-out


Lessons learned and take-away from this exercise

  • follow the right procedures and steps
  • ensure clear links between the steps
  • warning indicators + monitoring process have to be present
  • transparency in reasoning
  • goal: balance between structure & creativity
  • balance between conciseness (focus/summary) and depth (and NOT volume)
  • Define structure of presentation beforehand
  • Properly select colors and layout for readability and usability
  • The class presentation was the worst one due to lack of consistency and structure. It is therefore very important for the whole class that we have coordinating role to ensure a successful project


Group 4 - "Group For Waikiki"

Dear fellow strategists -- here are our picks from Daniel Erasmus' DTN site (go to Scenario Thinking/ Student Projects):


BEST IN CLASS: Distance Education 2011 (1996): 36 out of 40

WORST IN CLASS: Health in 2010 (1996): 15 out of 40


The eight criteria we used for evaluating the scenarios were:

Content Analysis

  • Relevant use and critique of theory, academic references, literature, traditional beliefs
  • Analysis of the issue with personal insight
  • Discussion and logical development of arguments
  • Quality of the section about scenarios
  • Other options/new directions for thought for the public
  • Synthesis of material

Design & Gestalt

  • Aesthetics & Consistency: Aesthetics, colors, size & font of text, choice of pictures, drawings, look & feel
  • User friendliness: Use of a site map, working links, clear navigation directions

Rating scale for each of the criterion

5 = Excellent, 4 = Very good, 3 = Acceptable, 2 = Meets minimum requirements 1 = Insufficient

Total possible score: 40


RANKINGS OF 10 SAMPLE SCENARIOS, LISTED BEST TO LAST

1. Distance Education 2011 (1996): 36 out of 40 (rated by Lucia Nedelcu)

2. Genetic Revolution: 30 out of 40 (rated by Mari Smith)

3. Relationships 2020 (1997): 29 out of 40 (rated by Claudie Chaumette)

4. Crime 2015 (1997): 25 out of 40 (rated by Eser Torun)

5. Interpersonal Communication: 23 out of 40 (rated by Spencer Rosen)

6. Free Internet Service Provision 2003 (1996): 20 out of 40 (rated by Spencer Rosen)

7. Branding 2005 (1999): 19 out of 40 (rated by Eser Torun)

8. Leisure 2010 (1999): 19 out of 40 (rated by Mari Smith)

9. Internet Banking: 18 out of 40 (rated by Lucia Nedelcu)

10. Health in 2010 (1996): 15 out of 40 (rated by Claudie Chaumette)

DETAILED EXPLANATION OF RATINGS & WRITTEN REVIEW OF SCENARIOS


Group 5
We are: Edo Avraham, Lars Eriksen, Kentaro Kodaka, Taro Honda, Daniel Perez Whitaker Why couldn't ANYONE post this link!

Scale used: Between 0-100, with 25=insufficient, 50=adequate, 75=good

The Information Society Grade: 75 Reasons: It has some very bold statements about the future. Of course they will not happen because their scenarios were for 2005, but quite a few situations are mind opening, Clockwork Orange like.

Telcoms 2003 (1996) Grade: 70 Reasons: Well coverage on PEST. Technology and industry insights such as extinction of AT&T / IP phone -Concentration of governmental regulations which is the largest constrain to telecom companies It should have been structured and mentioned step by step (ex. driving forces / uncertainties) and should have researched more on IP phone services and its superiority so that conclusion may be different. As a whole it’s still too simplistic. The two main drivers used are inadequate and cannot be selected as the only ones because of their limited application (i.e. User Demand and Level of Restriction). Many of the downsides mentioned view it in a 1996 technological limitations which not only in hindsight could have been ignored for the requirements of the exercise. Analysis is too naïve (cyber terrorism has been in existence without these conditions.

Crime 2015 (1997) Grade: 65 Reasons: A good composition of drivers (across most aspects). Sensible conclusions that was reduced to two scenarios with some plausible perceptions. Better

Telecommunications 2015 (1997) Grade: 60 Reasons: Well researched on PEST exhaustively. But it should have been structured and mentioned step by step (ex. driving forces / uncertainties). Less exhaustive coverage on PEST in scenario planning phase if seeing 2015, especially seamless services by mobile and ISP providers should have been mentioned.

The Leisure in 2010 Grade: 50 Reasons: Very well structured and developed but the effect of Internet on society and leisure was a bit exaggerated.

Delivery of entertainment 2006 (1996) Grade: 35 Reasons: The guy that wrote the whole scenario texts in UPPERCASE should be shot. Impossible to read. Content assumes technology as the main driver, while social and economical changes are ignored. The basic Porter’s 5 Forces Model is ignored as a basis for foretelling the main drivers for the changes. Read without refinement its mostly preposterous.

Food Retailing 2006 (1996) (Scenarios for supermarket retailing and the Internet) Grade: 35 Reasons: Discusses supermarket retailing exclusively through the Albert Heijn example. Process is clear but results are with such small differences that it’s either not ambitious enough or there is just too little breadth of variations. The original title “Food Retailing 2006” has a lot more options besides than focusing on AH. Not properly proofed shows lack of professionalism.

Group 6
Ranked in order of preference (from best to worst):

Genetic Revolution Very current and important topic for the future, well researched and a clear structured presentation. Most comprehensive scenario analysis. Good analysis of the driving forces, nice diagram.

Telecommunications 2015 Good presentation style but unfortunately not well structured although we did like the sailing analogy and graphics. Extensive analysis of the PEST driving forces.

Childhood freedom in 2020 Very unique although it needs a diagram. Quite hard to navigate. Rich in content but lacks an introduction.

Leisure 2010 Dull website lacking in-depth analysis as well as poor organization and presentation.

Organisations 2020 Another very dull website with broken links and hard to find the information. Overall scenario seems dubious.

Crime 2015 Very mediocre. Not very aesthetically pleasing – poor presentation although it has a good analysis of the drivers.

Retrieved from "https://www.scenariothinking.org/index.php?title=Scenario_Quality_Ranking&oldid=12359"