This morning I was listening to my favorite dutch radio station BNR (Business News Radio). They were talking about the statement of president Obama on the terrorist act on christmas day 2009 (see CNN / Daily News NY).

He stated that the “this was not a failure to collect intelligence. It was a failure to integrate and understand the intelligence that we allready had” (see USATODAY). “The bottom line is this: The U.S. government had sufficient information to uncover this plot and potentially disrupt the Christmas Day attack. But our intelligence community (red. CIA) failed to connect those dots, which would have placed the suspect on the no-fly list.” (See Politico). “The US government had sufficient information,” he said, “but our intelligence community failed to connect those dots…. That is unacceptable, and I will not tolerate it.” (See CSMONITOR).

In these times of realtime data warehousing and data mining I must conclude that again the childs mind is the best way to explain things to people in the (Oval) office and Pentagon. Visualise Data Mining and turning Intelligence / Information into Knowledge (and eventually Wisdom) can be done by using a simple drawing…

Turning the collected data into information, and using this information to gain knowledge about certain passengers, to eventually get this wisdom to forecast possible terroristic people, is as simple as connecting the dots / data. It’s the relation / connection / glue that ties them together and gives the bigger picture. In the future this way of working might lead to recognition of patterns (data mining …) and predictive analysis about the data – information – knowledge (see DIKW).

Bookmark to:

Could be the customers network I thought, but … on the phone there was no TWITTER either…

GOOGLING : Twitter and Hack

Found : THIS LINK

Looks like we need to use our own BLOGS again and microblog via WORDPRESS

==> UPDATE:

Twitter is working again… or is it a Iranian Hoax Frontpage to steal passwords….

Bookmark to:

One of my colleagues in the Business, Ronald Damhof, posted on his BLOG a reaction to this POST (see his article). His view is that the business metadata is data, just like the ‘normal’ data regarding e.g. Marketing or Finance. It could even be handled like the ‘normal’ data, and can be stored in a Data Vault-like model (TEMPORAL, EXTENSIBLE, NORMALIZED). I agree with him on that, but one of the question that remains is how these information streams should be linked ?

We are able to built Data Vault models for the Business Intelligence needs, and visualise the CSF’s and dashboards for the end users. We can even built Data Vault models for storing the Business Metadata, which in fact brings up a memory to an other project of mine, where the customer wanted to store this Business Metadata in a central spot. I have done an toolinvestigation (very small), and we found out that the Microsoft Sharepoint Solution was the cheapest and best-fitting solution, since the other reports where also built towards a Microsoft Sharepoint Portal (Using Cognos Cubes / Reports). Unfortunately this business metadata was not modelled in the data vault way, so … the solution was not used (until recently they started to ask for this again). Writing this experience I come up with an other thought / insight / question; how do we handle unstructured data in the data vault model?

So there are multiple questions in my opinion to answer;

1. Do we need to link the Business Metadata to the ‘normal’ data?
2. If the first questions is answered positively, how would we relate the Business Metadata to the ‘normal’ datamodel?
3. How does the Data Vault model cope with unstructured data, e.g. WORD, PDF, Pictures, Sound, etc.?

To answer the first question from my own perspective; I think it would be wise to do so (I think Ronald agrees with me on that). It gives context to the ‘normal’ data and provides valuable information about ownership, maintenance, definition changes, etc. => Data Lineage: How did the element I am looking at originate from the source after all.

The second question is very tricky in my opinion. If we use for instance the RECORD SOURCE as an example, we are not allowed to LINK to the SATs, where the RECORD SOURCE is used, also linking to the LINK would result in a recursive LINK (the RECORD SOURCE of the RECORD SOURCE of the LINK…). So, if we follow the discussion about RECORD SOURCES on this BLOG and the BLOG by Ronald Damhof, we conclude to use the natural key for RECORD SOURCES instead of LINKING to a RECORD SOURCE HUB. Thinking about this, I conclude that the DATA VAULT model for the ‘normal’ data, is a source in itself for the Business Metadata for this item. If we want to know more about the RECORD SOURCE, we just create a stand alone HUB in an other DATA VAULT model and LOAD it with all unique natural keys in the DATA VAULT model for the ‘normal’ data.

Bookmark to:

This post will zoom in on this simple example of how datavault can be used to visualise simple things and how simple things can be used to visualize datavault techniques. The question I started with was ; ‘Any one want coffee???’ . A simple question, but the answer is NOT that simple. It visualizes the effect of business keys used within a company and how different people use different (business) keys for the same goal; a nice cup of fresh coffee…

* “I’ll have a black coffee please.”
* “I want a ’11′ please”
* “For me it will be a ’0002′ please”
* “I’ll have a STRONG black coffee, please. An ’11111′ please”

Al these answers are used as business keys by the people, when I get my first cup of coffee in the morning. They all use different codes for providing me the same information and identify one cup of black gold. No Sugar, No Milk, just the black liquid.I just use these codes to provide them with the required cup of coffee.

But not all ‘systems’ are perfect. What if I want a cup of coffee less strong? Or what if I would like Milk or Sugar in my cup of coffee? The coffee machine only takes two interface;

(1) The selection method
(2) The short code method

The selection method is just simple selection of the requirement (e.g. less strong coffee, with lots of sugar and a drip of milk). It takes the user a lot of interfacing; selecting the specified liquid, changing the standard to the custom made cup of hot gold. Yes, you need to have an interface, the ‘user’, to translate/convert the requirement to a combination of keys, so the machine is capable of handling your order and providing the right cup of coffee to the right person.

On the other hand, the user can also use the short code. This has a BIG advantage, because the ‘user’ doesn’t need to have an interface, he or she just types in the code and gets what the customer has ordered. The translation / interface is at the customer (source system ?!?!).

Both methods are providing the product to the requestor. It is like datavault modelling, one person uses the social security number to identify a individual, an other person uses an userID and the third person just uses the individuals name, but the all identify the same individual with their different (business) keys.

So far for the business key visualisation by using a normal vending coffee machine. Next time I will do the tea example for you readers

Bookmark to:

But today I had a PERSONAL BRANDING training by Tom Scholte at our new office in Nieuwegein. A very nice training for people like me. People who are not that good in selling themselves on the internet, phone or by email.

What can one do to enhance your own PERSONAL BRAND? Simple ; Be visible. Work on visibility and let, any one who cares, know what you’re good at or which achievement you have reached. This BLOG post should be in line with that goal(s). I just want to share a phrase from a DUTCH magazine (KEK Mama*) with you, that I though was interesting in these times of recession;

Recession rhymes with depression,

but it also rhymes with progression !!!

Today I made some progression on my PERSONAL BRAND. Thanks to Tom Scholte and his training about PERSONAL BRANDING, I realised I had to just do it all myself and listen even better to what people have to say about me as a private person or in business.

* As of May 14th, 2009 my family contains one extra boy. Pjotr was born that day and young elders read stuff like KEK Mama

Bookmark to:

First I must admit that reading this posts was a recognition of a lot Data Vault principles. Ronald is a true Data Vault adapt in that way (Dan will be proud of you I guess, but don’t mistake the words with the archormodelling principles ). That makes it funny, but makes me wonder as well; are you using the Anchormodelling principles on your job, or are you using the DV words? Why did you choose to use these words in the BLOG, if you had the opportunity to use DV words as well…

I fully agree with Ronald that the T should be after the L of the Enterprise Data Warehouse. Like Dan would like to say, “I want the single version of the fact, because there is NO single version of the Truth”. This is because every person might have a slightly different approach to the same e.g. KPI or subject. Registrations / Operational data on the other hand don’t have that kind of intepretation / context problems. The Operational System just has to work 24/7 for the Business and e.g. firstname or DOB (date of birth) have no other meaning in diffent contexts.

I also recognise the nice architectural approach taken by Ronald on the EDW+ layer/instance. From the source landing area, data is flowing towards a data distribution and staging layer and consolidated in the stable layer. I see the staging layer as the source structured layer where the first step in Change Data Capture is done, before it is consolidated in the stable layer (which holds all unTransformed data). To have re-usable pieces of information, you could indeed use an ETL+ layer. This way we would create a efficient and effective layer which would combine the SINGLE VERSION of THE FACT with the GENERAL VERSION OF THE TRUTH !!!

The only thing I disagree on with Ronald is the part about the tools. When creating the layer model you will see the target (DV) model is almost always the same (at least for the HUBs and LINKs), which means they can be generated by ETL tools if configurated the right way. We could put the pressure on the developers of the tools to create these DV templates, but the most ETL developers in the field should have at least one or two DV templates for e.g. SAS / Informatica / OWB / AbInitio. This process untill the EDW (or even EDW+) isn’t that difficult it needs nifty ETL tools, it can even be done with the simplest SQL scripts i.m.h.o. The BIG T between EDW (or EDW+) and the datamart(s) is where the ETL tools deliver their strength for creating individual T processes easily by using the Graphical User Interface.

Maybe my vision is something to discuss in a nice new topic on Business Intelligence and Data Warehousing. In my opinion we could ‘build’/develop standard ETL processes for contributing to the DV thought of Dan Linstedt and benefit of these standard templates for Data Warehousing in the 21st century (using the Data Vault principles)

Bookmark to:

This is the reason that I missed a BLOG item by Johan van der Kooij (http://biconsultant.blogspot.com), who wrote about tagging for meme (http://biconsultant.blogspot.com/2008/04/tagged-for-meme.html). Since I have been tagged for meme, I will now continue the BLOG topic, but first the rules…

• Each player starts with eight random facts/habits about themselves.
• People who are tagged need to write a post on their own blog (about their eight things) and post these rules.
• At the end of your blog, you need to choose eight people to get tagged and list their names.
• Don’t forget to leave them a comment telling them they’re tagged, and to read your blog.

Some much for those rules, now let’s get on the real fun part, telling something(s) about myself;

1. I have started to work at DIKW Consultancy BV since april 2008, after I left SAS Institute (1996-2000), EOM Data (2000-2004) and Capgemini BV (2004-2008) => Do you see a trend?!?!?
2. I have a personal BLOG (http://www.smetser5.nl) aswell which I try to fill with my experiences outside my professional life as a Consultant in Business Intelligence. You should see some of the stories about my life there!!!
3. I am a competative sailor. I have been sailing since my youth (14-15 yr.) but not intensive. At the moment I have the possibility to sail on a BM 16m2 (recreational), and Dehler 39 (competative). Before I was sailing on a ILC 40 and MaxFun25 => The last one is a great boat to sail !!!
4. I have a wife and 2 children, which all have names which are all related to water and wind…
5. I am an introvert person, but if I get to know you… You will really get to know the extrovert side of me.
6. I am a hobby photographer (http://photo.smetser5.nl for my photoblog, which I haven’t updated in a while….)
7. I am a roundtabler (http://www.roundtable.nl, http://www.roundtable.name/, http://www.rt168.nl) in Lelystad.
8. I am an European….

Now I have to tag a few people that should continue this BLOG MEME …

• Richard Rombout (http://rrombout3.hyves.nl/)
• Jorgen Heizenberg (http://www.biguru-online.com)
• Wouter van Aerle (http://wobi.wordpress.com)
• Bas van Raaij (http://www.intelligent-business.nl)
• Erwin Hendriks (http://www.semseotips.com/)
• Larry Eiss (http://blog.larryeiss.com/)
• Dan Linstedt (http://www.b-eye-network.com/blogs/linstedt)

Enjoy the Blog Rolling …

Bookmark to:

 First let’s rephrase his questions;

• Why keep history of the Record Source?
• What should be done with the Record Source for LINKS and SATS, because the Data Vault doesn’t allow you to connect SATS to HUBS and vice versa?
• What about the LOAD strategy? Do you want to load the RECORD SOURCES while LOADING the HUBS?

To anwer the questions, I must admit I had some good readings about both the Data Vault and the Banking Data Warehouse Model (BDWM), and most are positively answered in the sense that Rob is right asking me these questions. Let start my monologue then…

I wanted to keep history in the record source because it could provide me with the information of releases and possible defects that originated from these releases. It’s more the descriptive information I want to store in the SATs, aside from the history it creates for my source system. Administrators could change. Locations could change. SourceType could change (e.g. Oracle => Teradata). But these are all metadata and perhaps do not belong in the Data Vault Model. Let’s ask the question to the user / customer and let then decide (I got the feeling it”s only maintenance that can benefit from this information, so maybe it’s better to skip this requirement…)

Rob is absolutely right about the SATs and LINKs connecting to the RECORD SOURCE HUB. Especially the SAT linking to the RECORD SOURCE HUB will give a lot of unnecessary lines in the data model and even a recursive relationship, because the RECORD SOURCE of the RECORD SOURCE for the RECORD SOURCE HUB… You get the picture

The last questions is about the LOAD strategy for the RECORD SOURCES. Since the LOAD strategy for the HUB, LINK and SAT contains explicitely the mentioning of the RECORD SOURCE in these entitities, I had the idea that the RECORD SOURCE could be loaded in parallel, but since there is only hardcoded mentioning of the LOADing RECORD SOURCE, the parallel LOADing of these RECORD SOURCE HUBs is pure crap!!! Hardcoded => LOADing… (Stupid thinking of myself). It’s only the METADATA I mentioned before that’s interesting to be LOADED, but that can be done sequential instead of loading it parallel with the HUBs, LINKs and SATs.

I think my only conclusion can be that the suggestion to use a RECORD SOURCE HUB is not a good suggestion. Let’s just stick to the hardcoded RECORD SOURCE for the HUBs, LINKs and SATs and maybe load / store the metadata about these Record Sources in a independent database or entity within the data model (regardingless of this being a Data Vault model or not).

=== CASE CLOSED ===

Bookmark to:

On this third day Dan was going to talk to us through the last sections of the technical stuff. After that we should be ready for the Data Vault test, which we are all going to take and hopefully all will pass this exam. But first we had to do some technical talking about the Data Vault.

First a little rephrase, again, of my open questions after the second course day;

Open Data Vault Questions

• How would Data Vault handle / cope with REVIVAL / REUSE of business keys?
• Why is there not a future dated ENDDATE in the SATELITES?
• How would Data Vault handle / cope with versioning of the EDW data model?
• How would Data Vault handle / cope with valuta conversions?
• How does Model Driven Architecture fit into place with Data Vault?
• Why are the IDs not unique across the model?
• Is the Data Vault contributing to the possibility to use Function Point Analysis?
• How does a METRIX satelite look like?
• How is a DELETE handled?

and then on to what we’ve learned and heard this day.

After a very short recap of the first two days, we started to work towards the exam at the end of the day.

I saw I had not posted this entry, but there is a nice picture in there, that should be shared with the community, so here’s the unfinished post. I hope to finish (get answers to all my questions) in the near future, but for now enjoy the visualisation of the CERTIFIED DATA VAULT EXPERTS in Apeldoorn – NL.

Bookmark to:

As I am a teacher as well, I read the article with big interest; “Should I now use FLASH movies for teaching the customers?” and “Do I want to use ActionScript for intuitive course design?” Well I must admit I have used Windows Movie Maker (powerpoint slides, YouTube Movies, Photo’s, other Capgemini movies) once for a presentation. Presenting on a Bang&Olufsen installation and using a projector with a resolution of 1024×768 on a white wall… I was astunnished by my results!!!

But was it the way of presenting my facts, they found interesting? Or was the fact that I used an unorthodox presentation tool the thrill? Perhaps they were even flabbergasted by the facts, Naaahh… In my opinion it was truely the power of interaction and attractive way of presenting that was perfectly matched to the target audience. A few young IT people, driven by gadgets and hottest technology, being served by a 15 minute movie…

Unfortunatly I lost the movie when a mobile harddisk of mine crashed, but I can assure you it looked very nicely. I will try and provide all the elements in the following list, so you might be able to create a nice movie on your own;

• Training material from the VOLVO OCEAN RACE (ABN AMRO 1 against ABN AMRO 2)
• Photo’s of the Capgemini building
• A Microsoft Excel sheet (inside a powerpoint slide) with the natural flow of consultants
• Funny pictures/movies/music from the internet to support the themes (Gordon & Joling / Sinterklaas / Politics)

Links:

• Black Betty (ABN AMRO 1) Promo
• Training / duel ABN AMRO 1 and ABN AMRO 2 (used including music)
• Bureau Renkema
• Boaten Klaas
• Gordon and Joling
• ZLASH
• Adobe Captivate (and some examples)

 Bottomline of this blog entry should be that everybody loves changes. Oldfasioned ways of presenting are replaced by new tools and abilities. Like overheadsheets were replaced by powerpoint, and now by multiple other tools (which are more intuitive and flexible towards the user), we might face a whole new way of distibuting and presenting Business Intelligence…

Was it not Dan Linstedt himself who started a whole new way of looking at a data model by using that fancy tool called RAPIDACE. If you ever have the opportunity to see a live demonstration, I would highly suggest you look carefully, because, walking thru a 3D datamodel is very nice !!!

Bookmark to: