Super Excited Because Agile Has Come to my Corner

UPDATE: 

With a move at the office, our whiteboard got "shut down" (there was no room for it). However, I found a great virtual whiteboard app I'm trying out called Realtime Board. It looks something like this (this is the demo):

PREVIOUS POST:

When I'm in a meeting and I hear words like BUY IN and APPROVE and SIGN OFF, I get really upset. Do you want to know why? Here's why:

The lower arrow is better

The BUY IN happens at the beginning in the first arrow. See how much failure is likely? 

The second arrow gives everyone a chance to get comfortable and "BUY IN" (if you MUST use that terminology) every time there's an iteration.

My new colleague and I are trying out Agile now because it just makes things far more transparent in terms of what we're doing and what we have to do. It is a better approach than having pointless meetings with too much management speak and not enough action.

Our board

Quality II: ADDIE application

From the last post:


User-based Quality, Product Quality and Manufacturing Quality all have characteristics that help me hone in on a quality model for learning products. 

Let's now impose ADDIE onto this trifecta of magical characteristics. 

ANALYSIS: In ADDIE, the Analysis is a needs assessment. In practice, this is when we ask ourselves the following questions:

Who is the learner audience? 
What is the mode of delivery? 
What subject coverage is desired?
What resources are available (SME's, documents, etc)? 
What are the constraints (it's usually time)? 
When does the project/module/learning piece need to be done? 

Garvin says: The characteristics that connote quality must first be identified through market research (a user-based approach to quality).

I'd say that each of these components of Analysis contributes toward high quality. You should be able to identify each of these as specifically as possible. In my experience, the more vague each answer is to each question in this list, the lower the quality. I'd love to back that claim up with a real test case or multiple test cases. I actually can. I can off the top of my head think of three distinct cases (not all my work, though one definitely was) where poorly defined audiences resulted in poorly defined courses that were not well-received. On the other hand, I can think of one course I handled that was well-defined in terms of audience and coverage. The course was well received. 

DESIGN AND DEVELOPMENT: Looking at these two phases, in ADDIE, we are basically talking about good design and good work. 

Design is the rigorous process of identify learning objectives for the material and for the audience, and then creating a storyboard or prototype to see how things fit together. Suffice to say, if you didn't clearly define your audience, obviously the learning objectives are going be more vague. If you did not limit your subject matter coverage, you may be dumping rather than designing. Avoid the dump. 

Good design checks:

ALIGN YOUR DESIGN: 
Define learning objectives and verify objectives with another party (SME ideally) 
Create a storyboard or outline
Does the storyboard or outline include graphical representation? 
Does the storyboard or outline refer continually to the learning objectives, and can you identify when and where? 

Development is where actual work gets done. There is a lot of room for quality checks during development. This falls into the iterative SAM/ADDIE/Agile feedback cycle. So feedback is definitely an important quality check that may benefit the end learning product.

ITERATIVE DEVELOPMENT:
How many feedback cycles has the work been through? With whom? 
Has the work been revised based on feedback? Why or why not? 

In summary, the product-based approach to quality rests on the idea that "high-quality characteristics" must then be translated into identifiable product attributes. Perhaps presence and absence of the attribute is good enough. Or maybe a combination of presence/absence and then some "standard" or level of the attribute would help to measure quality.

IMPLEMENTATION/EVALUATION: 
Implementation is what happens when the course is done and it gets delivered. Ideally you'd do a test run rather than dump the entire developed course into the lap of a trainer (or a student) and just hope for the best. However, this is how a lot of information is shared. Heck, I've done it myself. You dump everyone in the pool and see who can swim. 

This is not fair. Well-designed material is a representation of not only knowledge, but of your corporate brand/entity. Poorly designed material is, quite frankly, a downer.

How can we make the implementation and evaluation steps critical to quality? 

IMPLEMENTATION
Run a pilot (expensive) 
Test a component of the learning and evaluate it 
Train the trainer (train or communicate details)
Repetitive tasks such as uploading files have a quality check or are automated

EVALUATION
Are students able to perform the tasks set forth in the learning objectives? 
Are trainers able to convey the learning objectives? 
Did the hands-on work focus on the learning objectives? 
Gather actionable feedback - evaluations must be revised continually * 

The manufacturing-based approach to quality was and continues to be rooted in how well a product conforms to standards.An approach that ignores the "specification" will not result in a quality product. In the case of implementation, we'd have to see how well the material stood up to its intended purpose. The best vehicles for this test are 1) the learning objectives and 2) the hands-on material (labs). Other ways to make sure implementation goes accordingly is to make sure the trainers (if it's not you) are prepared. Finally, a cheaper, faster way to "test" implementation might be not to run the whole pilot but instead to take a chunk of the material and try it out before a major release. 

Poor Evaluation. The often ignored, youngest child in ADDIE. Evaluation can really bring everything back to the beginning by asking whether or not our specification - the learning objectives - were met? Why or why not? You can go back to iteration during design and development - did the design rely on feedback? How much? Is more needed? Was the design examined? Are we asking questions that actually tell us something about the content? Sometimes the comments are far more valuable than the smile sheets or ranking evaluations. 

Quality I: Old School Quality Concerns

A few days ago, I had a little idea about quality and instructional design, and I figured I'd really need multiple posts to deal with it. Here's the first of the multiple.

Garvin (1984) dug through his literature and summarized five ideas about quality, all stemming from different fields including marketing, economics, operations management and even philosophy. His goal was (I think) to form a more holistic concept for quality as it pertained to products. 

What he came up with was a list of definitions for quality. There are five of them and he has all the proper citations in his paper, and since this blog is really for my own mind wanderings, I'm going to go ahead and leave the citations out for now. 

Transcendent Quality -  proponents of this view claim that quality cannot be defined precisely; rather, it is a simple, non-analyzable property that we learn to recognize only through experience. 

Product Quality - Product-based definitions are quite different; they view quality as a precise and measurable variable. Quality reflects the presence or absence of measurable product attributes (which are assigned a cost, when analyzed economically).

User-based Quality - individual consumers are assumed to have different wants or needs, and those goods that best satisfy their preferences are those that they regard as having the highest quality. Personalized quality - this is very true today (2016). 

Even perfectly objective characteristics, however, are open to varying interpretations. Today, durability is regarded as an important element of quality (1984).  

Manufacturing Quality -  “conformance to requirements.” Once a design or a specification has been established, any deviation implies a reduction in quality.

Quality is defined in a manner that simplifies engineering and production control.  Efforts to simplify and streamline ((which are equivalent to reductions in the number of deviations) exist solely to reduce costs. This type of quality is applicable perhaps in software quality. 

Value-based Quality - According to this view, a quality product is one that provides performance at an acceptable price or conformance at an acceptable cost

Clearly, the "transcendent quality" definition is useless to me (shout out to the philosophers in the crowd!).So let me just ignore that. I am also going to ignore value-based quality because it is really saying that "cheap things have quality value because they are cheap". I just can't see how it fits into the learning product idea in terms of material quality. I am going to ignore it. 

Moving on, product quality makes a lot of sense: presence or absence of a measurable product attribute. Some of the examples include how much butterfat is in a good ice cream, or how many knots per inch make up a good rug. These are measurable attributes. The attributes themselves are arguably subjective, but Garvin assumes that most people would know the attribute as it pertains to quality.

Both the user-based quality definition and the manufacturing definition could work for me in terms of setting up my "learning product" quality idea. The user-based quality definition speaks very much to the current trend of product customization, which has a lot of value. The common person can have customized learning, just as he or she can have customized drinks at Starbucks. 

While manufacturing quality seems like a silly thing to bring up here, it actually isn't, because people talk about technology products in a manufacturing way all the time. I'm not saying I like it, but people do talk about standards and conformance and consistency a lot. There are a lot of boring, repetitive tasks in technology that ideally should be automated, but sometimes are not, or fall into a type of "mechanical" work that is similar to manufacturing. Large-scale file changes, uploads, changes.Even automation is subject to a quality check. So it's not ridiculous to consider the "conformance to requirements" in at least one of the ADDIE steps...ah yes. I'm trying to transpose some of these ideas into ADDIE....Mwa ha ha ha.

So in summary: User-based Quality, Product Quality and Manufacturing Quality all have characteristics that help me hone in on a quality model for learning products. 

The next post will clarify further...

Information Mapping - Overview

A colleague of mine sent me a few interesting links, so I'm putting them here for reference and commenting later. When I have time.

Horn, R. E. Clarifying Two Controversies About Information Mapping's Method Accessed 01/06/2016

Horn, R.E. (1974).  Information Mapping, Training in Business and Industry 11 (3).

Parsons School of Design Information Mapping Papers

ID and Quality: A Prelude

What is quality?

When we buy eggs at the market, we don't want a broken one. When we order food at a restaurant, we want it to taste good. A high quality experience in economy class is when the flight attendant offers you a glass of complimentary wine just "because you look like you need it" (I recently had this experience, and it resulted in my writing a glowing satisfaction report for the flight). 

What is poor quality? 

Every egg carton has broken eggs. The food is too salty in the restaurant. The flight attendant was not warm and friendly. All have different root causes, but each of these can make or break a quality situation.

How do we define quality when it comes to instructional material? How do you know if what you have produced is high quality before you delivery the training?  

In order to answer these questions, I have decided to treat instructional material as a product. It is, in a sense, a product. It is painful to me, as both the daughter of academics and a trained academician, to admit this, but the plain fact of life in America is that everything can be productized. All our experiences are at some point commodified. This includes the very act of acquiring information. When you design instructional material, you are really designing a product. A learning product. 

Even though it annoys me, working on this assumption eases the search for answers because there is a mountain of literature and information about quality (but how good is it?). 

One seminal work that I'm going to start with is David Garvin's (1984) work titled, "What Does Product Quality Really Mean?". I am looking at this one closely because it is a review paper that summarizes ideas on quality (until 1984, obviously). One glaring point here is that in 1984, software quality was in its infancy. Therefore, software training was not even born yet. However, I'm going to argue that the ideas of quality, as a concept, are somewhat ageless, and that there is some hope that we can apply the ideas in Garvin's work to the present questions: What are the key attributes of high quality instructional material, and how can an instructional designer strive to product high quality material? 

My ultimate goal is to answer these two questions, and also produce a checklist that helps instructional designers determine whether or not they have met a basic quality standard. this is probably going to take more than one post!

References

Garvin, D. (1984). What Does Product Quality Really Mean?" MIT Sloan Management Review. 

For my Reference - Lots of Links!

eLearning Classic Texts & Reference Handbooks

If you aren't sure about what you are doing, check in this nerdy textbook (books have indices):

Clark, R. C., & Mayer, R. E. (2011).  E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning . John Wiley & Sons.

Kuhlmann, T. (2015).

Before You Begin...

at DevLearn!

Top 5 eLearning Acronyms (yeah, I know there are more but why start with Cognitive Overload (see below)?) 

10 eLearning Course Development Mistakes 

For the critical thinker: eLearning: Are we missing the point?

Best Practices

Storyline Best Practices

Learning Objectives and Best Practices

Tips to Give and Receive Feedback

When is Audio Narration Helpful?

Be Wary: Cognitive Overload 

What is Cognitive Overload?

Limiting or doing away with cognitive overload in your eLearning

Planning eLearning

Basic plan overview - this is a general article, but it has some good tips

Storyboards - lots of options

What is Storyboarding - We had a BrownBag Lunch about this?

There is a Lynda module about Storyboarding (ask Lisa IC for access) called Introduction to Storyboards

Storyboarding is a Total Waste of Time (Well, everyone is entitled to an opinion...)

Storyboard Templates - eLearning Heroes resource

Forum Discussion about Storyboards - eLearning Heroes resource

Agile and eLearning

Short definition here

Formalized versions of Agile for ID - works for eLearning and ILT

SAM - Successive ADDIE Model (ADDIE on speed) by Allen Interactions

LLAMA - Lot Like Agile Method Approach by Megan Torrance

Criticisms of Scrum

Graphic Design and Images for eLearning

eLearning Graphics - Seven Best Practices for Choosing Graphics (suggestion from the eLearning Industry Blog)

eLearning Images - Six things to think about before you select an image (e.g. photograph) for eLearning courses

Building your Own Graphics - Do I have to? Sometimes you do...

10 Tips to Improve your Visual Design

Elegant UI and Web Design Handbook

The Persona Problem - Everyone talks about it, but what are the critical concerns - cognitive economy and other generalizations..

 

 

 

PowerPoint - is there really a right way?

Until pretty recently, ID was a job that was mostly creating effective slides for presentations. To that end, there are many ID's who "know" PowerPoint. Heck, most business folks and professors know PowerPoint. Nowadays, kids learn how to use PowerPoint in junior high school. It is a somewhat ubiquitous tool.

However, it's come to my attention (not just now, but over the past few years) that PowerPoint is a largely underutilized tool. PowerPoint has developer tools (an Add-In). PowerPoint has numerous customization options. Because of this complexity, I started thinking about whether or not there are any standardized and accepted best practices out there for PowerPoint.

Guess what? 

There really doesn't seem to be anything really clear when I do a cursory web search (I'll keep looking). There are tons of tips and tricks. There are numerous "good" and "bad" examples. In fact, I really liked this webinar, which shows numerous slide "makeovers" based on basic visual design concepts. That could be one important element of designing a "best practice".

The next thing to consider is something that we've been grappling with at the office. Something that seems so obvious but is an error that has been repeated ad nauseum in corporations all over the world. It is that of the PowerPoint template. With changing versions of PowerPoint as well as employees rotating in and out of projects (employees with varying degrees of skill when it comes to PowerPoint), there is this phenomenon that I am going to call The Tempblend. It is what happens when lots of slides from different places come together in a single presentation. It's the melting pot of colors, graphics, typefaces and layouts. It is what you don't want when you have to go into a giant brand conversion or upgrade; I am willing to bet it happens EVERYWHERE.

While most corporate old-timers are probably aware of this issue, I personally had never been exposed to this level of detail when it came to PowerPoint. When you teach on your own at a college or in a classroom you have complete autonomy over your slides and your material. We are generally left to our own devices when it comes to slide management.

But, in a situation where there's turnover and varying degrees of skill and competency on a team, the Tempblend and other disasters are usually lying in wait. Is there a way to overcome these problems and improve slide management?

I did find one article about scalability which basically says, "this is a problem and there is no good answer." At least there's acknowledgment.

The only takeaways for me about this common ID problem are:

1. Be as consistent as possible with PowerPoint templates. A good template

• Use only the master slide(s) you need
• Name your new template's layout slides
• Use whatever Layout slides you have 
• Do not copy and paste slides from other places indiscriminately. The Reuse Slides option in PowerPoint 2013 guards somewhat against this, but if you were careless in the past, then the problems of reusing slides may still appear.
• Check for artefacts from old masters (View -> Master Slide) and delete unwanted slides in the Master.
• After an import, verify that Layouts are applied to slides. 

2. Gain some skill in Visual Basic in order to automate fixes. 

3. Look around and find tools to help fix the problems you see. Sometimes you have to pay, sometimes not. I found one tool that will fix image issues when you switch from 4:3 to 16:9.