Black Swan Society

Hi folks. Need help. One of our really important stakeholders is making it a deal-breaker that we somehow get a system that allows us to pull data from both the LMS and the ERP so that we can actually run some real business metrics to measure the effectiveness of our training. (Shocking, I know). Our LMS does not do this natively. Neither does our ERP. 2 questions: - If you have a robust database of this type, what are you using / what did you do? - How did you overcome the objections of the IT department (of which I am a member)? Thanks for any and all advice. Wendy

I'll take a crack at #2 first. Maybe that will help answer #1. The first thing I did was to see what I could pull out on my own, from reporting standpoint. No approvals needed for that. In fact, I think you'd be surprised to find out exactly how much information you can pull out as a report from the LMS and from the ERP. Now, it doesn't natively integrate this information if you're not in some integrated system. Even if it did, the reports don't necessarily tell you anything you want to know. So let's focus on a problem. What is a question you want an answer to? Maybe it's something like how effective a safety training course was? So to look at that, you might pull up a list of all the people who've taken your Safety course from the LMS (that should be pretty straightforward). Then you pull up from the ERP all the incidences of a related safety issue year-to-date. If you can view it by employee IDs, better. But the idea is to try and answer as muich as you can on your own for this one question you want answered. And... document the anecdotal level of effort involved in doing this. When you present the information to your people, the fact that you can do so much of it yourself anyway will take some of the fear out of the request, because they're obviously pretty well exposed already (chances are). When you can pull the data together, you'll be able to demonstrate the effectiveness of even crude reporting. Your trick then is two-fold: one, get them to support a tighter integration and relax a bit about the transparency; two, you don't want this to become your fulltime job unless you want this to become your fulltime job. Does this help at all? Or am I missing your mark?

Helps a bit. If only to break down the issue into manageable chunks. I know part of what the stakeholders want is a pre-integrated system where we wouldn't HAVE to pull 2 separate reports and crib them on an excel spreadsheet. Some groups are doing this already. What the big stakeholder wants (and, hell, what I want) is to have a system that does this automatically. Is there such a beast? This is where your advice to define what is absolutely needed (vs. nice to have) and document the effort will help if we have to build this puppy ourselves. Especially since no one is certain that our ERP can actually handle this type of interface. Of course, we've had this ERP since version 3 (we are about to roll out version 8) - and I don't think we've rolled out a new feature in forever. The political concern is that the IT group is already overworked and more than a little stressed. I think customization is unavoidable. We need the IT analysts and administrators buy-in. So I guess the next question is - is there something that integrates business systems and LMSs? Is there a way around having to construct this thing? (and no....I don't particularly want to make this my full time job. I'm a terrible programmer / database designer...)

I'm assuming your ERP system is either SAP or PeopleSoft, right? Your LMS? Probably doesn't matter. If it's not SAP or some integrated Oracle solution, customization is unavoidable if you want to actually get this stuff to talk to each other. I've not heard of a middleware solution that integrates both reporting mechanisms. Now that said, it's possible that a guy like Chuck Allen might be able to help (Chuck, @chucka_nc, is an HR-XML expert and he's into interoperability of HR systems). You're not the first to ask for this, but you're probably the first to be able to summon the collective intelligence needed to both identify what the problem is and highlight what can be done that looks like a repeatable solution.

ERP system = Banner (SunGard) Current LMS = SkillPort. Though we may also be looking at a Blackboard integration so that Blackboard is the primary front end. I have a feeling this is what we are going to be working with. Though I may ping the team to see how far we can /need to go to get something integrated for the appropriate metrics. A small test of the "real value" of training. Definitely putting Chuck's name in the back pocket. I have a feeling we will need it. Thanks so much for your help.

I'm a little late to the game (and hopped up on cold medicine so take what follows with a cautionary note), but perhaps you are overcomplicating the problem (or I'm not understanding the problem). If I understood Aaron's earlier response correctly, he seems to be looking in the direction of actually generating reports from the LMS or the ERP system (i.e. what canned reports exist, what tools do these systems provide to create your own reports). I think the problem with such a direction becomes identifying the 'owner' of a particular piece of data - i.e. which system is the master, which are the slaves. This is definitely an issue IT fights around. To alleviate some of these types of concerns, we were moving toward the data warehouse idea. In the warehouse, each system owns their particular pieces of data - they all just get centralized for mass reporting and analysis. So in a simple example, the relevant data from Banner would be ported to a database server at specific times (nightly, weekly, real time) depending on the needs for particular data elements. The same is true for the data from the LMS. These data ports are usually a fairly simple arrangement from an IT perspective so no new expense required with the exception of having a DB server capable of meeting your query needs. On top of that you implement some type of reporting solution - Business Objects, Cognos, Crystal Reports, home grown, whatever. Give people access to the data warehouse and watch them create their own reports. Most of these tools allow you to design a report and have the query executed at specific times. Copies of datasets or reports and usually be emailed or shared across the organization. As Aaron mentioned the trick is really in understanding the type of analysis you intend to perform. This has huge implications on how you collect data, how long you store data, and how rapidly that data must be refreshed in the warehouse. For example, we were pulling reports together that tracked a business specific metric for some period of time before and after a training class. We were also able to compare that data between the 'trained' group and a control group. We also merged business and learning data to identify people who were underperforming - i.e. these 500 technicians are responsible for 80% of all recalls. Let's dig into the repair history to identify what the root cause was and provide a very focused training program geared at reducing recalls by X%. Again, I'm a little fuzzy due to the cold medicine, but I think this is the direction you are wanting to go. Set me straight if I completely hosed something up.

John - what you say completely and utterly makes sense. I had a talk with some of my bowling buddies - one is an IBM Global Services guy who's been on a number of integration projects. The other has lengthy experience in the eLearning vendor world (though which one he worked with escapes me at the moment. Both of them said the same thing - data warehousing. My next move is a friendly conversation with our resident DataMarts guru. The Banner information we need is already in the tool. The next question is whether we can get the LMS information in there. As one of my friends aptly pointed out - the REAL trick is getting people to give up their information. Thank you so much for your help.

What do you think you can get from your ERP that will show "business results"? At first, I would highly recommend you avoid worrying about how you will pull things together and start with the real requirement. Chances are 80%+ that this will go away (at least at the level you originally described) once people realize the data you have available doesn't really correspond to what they want. BTW - interesting to find this resource and surprising that it's separate from twitter or other places to chat?

The example I was given: - Does the training program directly impact the amount of money brought in through fundraising. Apparently, the Development office for our university is already pulling fundraising statistics from our ERP. - How much $$$ Development Officer Y is bringing in. So...if Development Officer Y did the training program, participated and got decent scores, did the amount of $$$ go up? For how long? If Development Officer K got lousy scores and only did 1/2 of the training - is he seeing the same increase in $$$ brought in? Less increase? Are we seeing the same trends across all of the Development Officers? This is probably the most concrete scenario I've been given in my environment. -------------------------------------- Aaron, Koreen and Mark O put this in place soon after the Innovations in eLearning conference last year.

Okay, so let's assume you have a unified ID for each Development Officer in both the ERP and in the LMS. Then, the initial data you would want is: * Fundraising by Officer (ERP) * Courses and Scores by Officer (LMS) These should be easy data dumps. Import that data and do a very simple analysis (Access DB / Excel). You can do a first cut at this very quickly and then you take it around to see if anyone believes what you are finding. In my experience, there will only be marginal differences in most cases, but still likely some positive correlation.

You all may be right and that I am over-complicating the issue AND focusing on the wrong things at this juncture. Thankfully, there is a unified ID in both systems since we are using our LDAP for authentication into the LMS (one of the smarter things we have done). Right now - seeing what resources we have in-house to automate some of the import ---> excel work. As my director just pointed out to me via IM - "the requirements really are ideals right now. Let's focus on small wins - like the fact that the training groups are actually TALKING to each other." I suspect she is going to take on the "Expectation Management" piece of this puzzle as I try to work through this issue.