*update 2017-05-19, found this thread on reddit with a safer fix/workaround: seems installing the new Windows creators update fixes the issue, so would advise for that first!*

Just received great feedback/hint from Eric E Swanson (see here) -while applying the described workaround successfully  (great to know!). From his reply:

“…This problem happened to me when I used the Disk Cleanup utility and removed Old Windows installations and/or Temporary Windows Update files (not sure on the exact verbage, but it was part of the list when you click on System clean up)…”

I did a few tests on my own (yes, I had that problematic folder properly backed up, and a full OS partition image also, just in case) and…

Turns out a surprising operation that can cause the Windows Critical Service Failed boot BlueScreens is really…. Windows Disk Cleanup Tool…

Just tested and really something strange is happening with Disk Cleanup. First time I ran it, left the default few options (sorry didn’t proper screenshot that), confirm delete, wait a few seconds, nothing. Everything still the same on C:\Windows\System32\CatRoot\{F750E6C3-38EE-11D1-85E5-00C04FC295EE} 4.586 files from my last restore (note: these are driver signatures that Windows checks if drivers are properly signed)

Ok, I don’t give up easily…. 2nd run,this time selecting Clean System Files, then leaving the only default option checked, Temporary  Files. Also strange that it mentioned ~20GB of cleaning (that folder is around ~100MB total)

But this time a few files were actually removed from C:\Windows\System32\CatRoot\{F750E6C3-38EE-11D1-85E5-00C04FC295EE} . 

Ok a few files removed… strange, so ran it again, same options, selecting Clean System Files, then Temporary Files. Also strange that now seems to mark only ~15GB to “clean”. Swear probably it’s the last I use this tool…..

So, this time, after cleanup, these were the only files left: (67 files only!) Almost the entire driver signature folder completely removed:

So, testing the boot right? Back to the old boot BSOD/Critical Service Again (at least that makes sense!). Now much quicker to recover, as described in previous post, just pasted the files again from a backup folder (after disable boot option, drivers signature enforcement).

Boot again with the folder recovered, no more blue screens or issues with unsigned drivers!

Even strangest, tried several times after that, even with a few reboots, can’t seem to replicate it now using Disk Cleanup. Folder steady at the 4.586 files.

After all this, also strange than I got now ~5 GB (keeps changing?) yet to be cleaned no matter how many times I try. Meantime, I can confirm I have around more ~15GB free space on the win partition. What files has cleanup removed? Don’t have a clue and it’s kinda freaking me out….. tool from hell. (which btw remind me the BOFH immortal words… “Well, let’s see, you have 4 Meg available" …)

Searching online there are several help requests regarding this kind of Blue Screen , and even a few mentioning Disk Cleanup as a possible cause. (see below)

Didn’t found any official/tek details info from msft , seems lacking proper awareness & follow-up from msft support teams honestly (yes, if not already, someone should really be looking into this).

How can such annoying, frustrating boot Blue Screen  keep appearing online and being so difficult for users getting proper solution beyond the typical- format & start again? (seems also something is lacking on the startup logs/troubleshoot process, are we not in the age of big data?…)

So, hope this helps,(helped two of us any way )

Rui

Similar issues mentioned online

Reddit-Windows 10 Critical Service Failed BSOD reboot loop. Boot fine with driver signature enforcement disabled.
(mentions installing creators update as a fix, better one I would say)

All System32 drivers are suddenly unsigned in Windows 10
(thread where I posted the workaround and got the Disk Cleanup hint from Eric E Swanson )

Disk Cleanup causing Windows 10 to go into BSOD loop

Windows suddenly won’t boot, “CRITICAL_SERVICE_FAILED”

Windows could not load properly after disk cleanup

Windows 10: Disk Cleanup is causing a Blue Screen

OS is going to blue screen after disk cleanup

Started from Digits CPU Docker image here (amazing work with these Docker files by Kai Arulkumaran) and just added volume configuration for digits data & jobs.

So you can use, on Docker/Ubuntu :

docker run –rm –name digits-cpu -d -v $PWD/digits-data/:/data -v $PWD/digits-jobs:/jobs -p 5000:5000 rquintino/digits-cpu && \

docker exec -it digits-cpu python -m digits.download_data mnist /data/mnist

 

On Docker/Windows:

docker run –rm –name digits-cpu -d -v "%cd:\=/%/digits-data"/:/data -v "%cd:\=/%/digits-jobs":/jobs -p 5000:5000 rquintino/digits-cpu

start http://localhost:5000

docker exec -it digits-cpu python -m digits.download_data mnist /data/mnist

One issue you’ll probably have with docker windows data volumes is performance. Think there are some known issues regarding some operations on volumes on Docker Windows, which the mnist prepare script should be triggering (60k images).

CPU Train Performance

Using the GPU on the Azure Data Science VM NC series model full train for 30 epochs is around 1-2mins, using the CPU version like above I had to wait ~19 minutes. So 5 minutes won’t be enough!

Image docker hub/github

https://hub.docker.com/r/rquintino/digits-cpu/

https://github.com/rquintino/ml-dockerfiles

Rui

We’ll be using Nvidia digits, that wraps Caffe deep learning framework, will do all the heavy lifting for you.

If you don’t have an Azure Subcription you can get a trial easily

https://azure.microsoft.com/en-us/free/

Create the DataScience Linux VM

be sure to select HDD and NC series VM (with K80 GPUs)

Wait a few moments until it’s available for remote connections

Connect to the Linux VM using SSH (ex: putty, mobaxterm)

you can run dvsm-more-info to get a guide of what’s available, pretty much everything you need

Start Digits Docker container with nvidia-docker, download & prepare mnist dataset

shell code: (also available here)

sudo usermod -aG docker $USER && \

mkdir ~/data && \

mkdir ~/digits-jobs && \

nvidia-docker run –name digits -d -v ~/data:/data -v ~/digits-jobs:/jobs -p 5000:5000 nvidia/digits && \

docker exec -it digits python -m digits.download_data mnist /data/mnist

this will:

1. add current user to docker group so you don’t need to sudo constantly to use docker
2. create data and digits-jobs host folders for persisting data and digits jobs outside of the container
3. run the digits container (default gpu based, exposing digits port 5000, using nvidia-docker as to use the host GPUs)
4. executing python script inside container to download & prepare mnist dataset

God, I love Docker….! (thank you Pedro Sousa, owe you for life! )

SSH Tunnel for digits http/5000

Now, digits has no real authentication so I suggest you configure ssh port tunnel (don’t expose this endpoint!), so that you can open http://localhost:5000 on your laptop and will open the remote digits endpoint running on docker container running on azure data science vm host .

Open Nvidia digits on http://localhost:5000

You may have to wait a few seconds, and then should open on your local browser.

You can follow the guide here, no code needed: Getting Started (NVidia Digits) 

No more code required to train your first deep convolutional network with GPUs and get 99% accuracy on mnist handwritten digits dataset. Should take only a few minutes

note: path for mnist data is /data/mnist/train for use when creating the dataset, you don’t need to download the mnist data again

Some additional screens

Exploring dataset images

Training default LeNet network

Network config

Digits built-in network visualizer

GPU training (1 tesla K80)

done, ~1min 37 seconds

testing trained model/epoch

~

it works!

activations, filter weights,…

Persisted mnist data and digits jobs on host vm

so you can stop docker/container if needed, later start a completely new docker container and continue where you left off

Having shown digits nocode approach here, the really truly inspiring is getting to know how this convolutional neural networks work and how they learn the filters on their own (they learn the $”%#@#”"&% reusable filters/weights! then how to use them to build other filters…and so on). Absolutely amazing.

If you’re interested few good sessions.

Tensorflow and deep learning – without a PhD by Martin Görner 

How Deep Neural Networks Work- Brandon Rohrer

Take care!

Rui

*update 2017-05-19, found this thread on reddit with a safer fix/workaround: seems installing the new Windows creators update fixes the issue, so would advise for that first!*

*update 2017-05-19, please read the part II for additional info and possible issue with Disk Cleanup tool: Disk Cleanup and Windows 10 boot Blue Screen / Critical Service Failed – Disable drivers signature enforcement, Unsigned Drivers *

Please note: It’s been stable and no problem with restarts for a few days now, but not saying it’s a permanent fix or better than a clean install. Will update if something changes. (probably something on my system is triggering this and will happen again)

My advice also before testing any kind of changes or emergency repairs, keep calm, don’t make it worst, backup the data you want safe before anything else.

My setup

• Dual boot, default Win10 enterprise and an old Win81 legacy boot partition I still use occasionally
  (ex: when I get a recurring Blue Screen on boot !), no bitlocker
• Separated Data drive, with bitlocker
• Win10 Build 14393.1066 Version 1607
• Single SSD, 3 main partitions

Symptoms

• On restart and before my dual boot selection screen I get the Blue Screen,
  detail message is Critical Service Failed
• 
• Suspect this usually happens after major updates, I have the bad habit of never restart and always hibernate…
• The system is bootable if you choose the “Disable drivers signature enforcement” on the troubleshoot / AdvancedOptions / Startup settings screens (usually option 7 when you restart)
• 
• After restart with “Disable drivers signature enforcement” windows works pretty much as normal
• But every restart you will get the same Blue Screen as this option is not permanent (nor advised), and I end up having to do two restarts to enable the option again, better than nothing but annoying
• Running sigverif  tool (after booting with  “Disable drivers signature enforcement” to get Windows working) showed a huge number of “not signed” drivers??
• 
• in Event log (Application and Services Logs > Microsoft > Windows > CodeIntegrity > Operational I see a lot of errors when windows tries to load driver signature files needed, seems suddenly all drivers are being marked as not signed and Windows won’t allow it, except for enabling the “Disable drivers signature enforcement” boot option, also errors pointing to files I don’t have in C:\Windows\System32\CatRoot\{F750E6C3-38EE-11D1-85E5-00C04FC295EE}
• 
• Drivers then get marked as not signed! not good…
• 
• When compared to similar setup from another win10 same build, the number of files on my C:\Windows\System32\CatRoot\{F750E6C3-38EE-11D1-85E5-00C04FC295EE} is very low (~1600 vs ~6000 files in the working win10), only two files starting with M… on my folder?
• 

What worked for me (use as is, as a last resource, no guarantees whatsoever, do your backups!):

After checking my win 10 build, got a zip with all the files in C:\Windows\System32\CatRoot\{F750E6C3-38EE-11D1-85E5-00C04FC295EE} from a “healthy and compatible donor” (thx António) and pasted the missing ones on my system. Being cautious not to overwrite any file just in case and doing a backup of my folder (obviously!).

check the build numbers, *probably* they should match your win build version, ex 14393

Restart, now the boot selection appears, choose win10, everything working. sigverif tool now only complains about few files that seem not critical, previous Eventlog CodeIntegrity errors also clear.

Now, what I really have a hard time to accept is why do we contribute with so much data, diagnostic & logging information these days, and still no sign of minor “windows data intelligence” suggestions to help with a problem that for sure I’m not the only to experience…

Thx also to Pedro Sousa and José António Silva and our Help Desk for their tips & guidance.

Yet again I skipped that clean install!  

Some references with similar problems

Critical_service_failed BSOD restart loop windows 10

All System32 drivers are suddenly unsigned in Windows 10

(Which reminds that one of these days I probably should share some thoughts on bitlocker, efs and …backups, do your backups! and be sure to test them!)

Rui

Must confess I hadn’t so much fun in a while working with data, stats, code & dataviz, been an amazing learning journey, really highly effective on consolidating ~more than a  year of almost exclusive deep dive/personal quest in data,stats, R, machine learning (and yes, some math involved…).

Have to say  almost as equally frustrated missing the top 10….again, 6th on public leaderboard, 13th on final rankings. Far from one of my  goals, actually winning the competition. And trust me, I gave it all, to the last mile.
(ah… and yes, prizes would be great help on research funds )

I had got a very lucky 34th place in previous brain signals competition, but that one was completely out of reach, due to the knowledge needed on signal processing/ECoG, gave up midway really)

So, I decided to take this one very seriously as I knew it would be the only way to maximize learning, both on competition data, ml process and everything data/stats/ml related. I would only stop when there were abs nothing more I could do, and so it was.

Other goal, ensure that DevScope ml minds could maximize learning also, have some fun, hopefully get into winners, or at least top 10. Very happy to see that we’re all top 25. Huge congrats Bruno & Fábio. (still couldn’t  get most brilliant mind at DevScope to enter this one though, hope better luck next time…. )

I put so many hours… days, weeks on this one, if there hadn’t been so much learning involved  (and personal enjoyment)  I would classify this as my most unproductive period… ever.

So, I’ll publish as much as I possibly can in upcoming blog posts, what worked out, what didn’t, code, tips, insights that still intrigue me, from Azure ML, to Power BI, R, caret, xgboost, random forests, ensembles, learning with counts, even some python. Also noted as much as I can on code files, OneNote was huge help, it’s great now to look back and have all this.

Other goal was to pursue both native Azure ML & custom R tracks to compete, but preferably win with Azure ML native modules (as I knew few top competitors would be all R/python). Still believe that using AzureML modules, no code required, would be enough for top 10-20 rank. But also believe, with current azureml capabilities,  if you want to win, you’re better off using native R/python from the start.

That’s relevant, also relevant  that top 3 winners submissions use base R, xgboost,  all features, no AzureML native modules (azureml teams, some hints here…)

Still, it isn’t over yet, as now I can look back and try to understand what could I have done differently and learn from top submissions. How much randomness is involved in a competition that was so close like this one? bitten by excessive variance on my submissions? multiple comparisons? Can an ensemble of the best techniques and models from top submissions improve the score we have to date? what more insights can we get from competition data? submissions data?

Awesome journey, so much still to learn…. hope to share as much, stay tuned!

Rui

(just the tip of the iceberg really…)

online PowerBI /EDA Companion

Cortana Machine Learning Competition – Womens Health Risk Assessment – Power BI Companion

Small companion for anyone participating in the Cortana Machine Learning competition – Women’s Health Risk Assessment

The Power BI file allows for easy data profiling over the supplied training dataset, overall patterns, data quality, missing data, feature selection. Note: you’ll still have to build the models though!

Although the current version of Power BI Publish to web doesn’t allow  R visuals, they’re highly recommended (ex: density plots & others), and can easily be added to the data/column profile pages.

So what are you waiting? You can be running your first submission in minutes using the online tutorial. Then it’s up to you.

http://community.powerbi.com/t5/Data-Stories-Gallery/Cortana-Machine-Learning-Competition-Womens-Health-Risk/m-p/61588

The Story of Dr Ignaz Semmelweis – “Savior of Mothers”

This sample PowerBI workbook tells the real story of Dr Ignaz Semmelweis. Year is 1846, Dr Semmelweis begins his 3 year work at Vienna Hospital maternity clinic, and is immediately  disturbed  with the high incidence of puerperal fever (also known as "childbed fever") in Vienna Hospital, causing  extremely high maternal mortality between 1823-1850.

Semmelweis was one of the first data “detectives” of our time, using data and statistics to test his hypothesis and theories.

This workbook allows you to explore the data collected by Semmelweis through those years and shows typical exploratory patterns we can use to hint on important information and possible causes from observational time series data.

http://community.powerbi.com/t5/Data-Stories-Gallery/The-Story-of-Dr-Ignaz-Semmelweis-Savior-of-Mothers/m-p/62206

You can see the video walkthrough here:

https://www.youtube.com/watch?v=aDURDaFaM0I

So… finally dumping a blog pots that formed on my mind going back several months (cof, years…) now… Anyway, today is even more relevant than then, so…
 
Warning: there won’t be fancy templates, crazy ideas,  strange tools, insights, snippets or new downloads on this post. in fact it will be a completely useless post for anyone reading I guess, I just need to get it out.  Don’t have a clear objective, or clear picture of what will come up. Also I won’t be slightly concerned if it’s too short, or too long, I won’t massively edit it (ok, slightly…). Fellow reader, I’m not writing for you today, I’m sorry.

Still, A-post-must-be-written.

So, how to start? How did we get here so fast? Seems only yesterday we were seeing this new Power BI Dashboards screenshots shown at WPC 2014. ~7-8 months after seeing Power BI GA then already a immensely powerful & stable Enterprise/Self-Service/even Personal BI platform, soon after that our first major project with the technology is online.

I’m not easily surprised, and yet time & time again I was completely baffled with what Power BI teams came up delivering, beyond user & community most requested features & needs, a lot others I honestly didn’t knew I was missing, and yet I can’t live without today.

Such an amazing journey… not in my wildest dreams, honestly I would say we would get here. And boy did we need it. Back in 2013-2014 remember we were getting very uncomfortable, mainly for the medium sized/enterprise BI scenarios, at least for any BI team that only works with Microsoft BI stack: powerful but completely stale OLAP engine, no clear roadmap,  failed products ("Planning" anyone?) , acquisitions that ended up bringing no value in the long term, huge number of disparate tools with poor integration, pieces that didn’t fit properly together aside from very superficial demonstrations. (we deliver solutions we hope can add value, not fancy toys that work great in demos)

The nagging feeling that just selling the latest tools was more important the adding value. I’m still a big believer that revenue is a side effect from value we bring for customers. Add value, then capture some of it so you can add more value, repeat.  You can make the other way work temporarily but it won’t be sustainable in the long term, and even if it were… that’s not the point right?   That was the  feeling honestly, yes, we had some great tools (heck PowerView  on SilverLight was great for its time, we couldn’t have Power BI today without “project crescent” right? :)  ), but still msbi kept lagging dangerously behind new vendors like Tableau, Qlick, etc in several key areas.

Something was missing, purely speculating here. Maybe that came finally with Nadella, also James Phillips strong vision, bringing the needed kind of startup mindset to Power BI vision (reminder, speculating here!).

Don’t have much visibility on Microsoft Power BI internal journey, for years know I’ve been inspired with the lean startup, customer development movement, mindset, and from the outside at least, seems a lean startup perfect case study. Hope one day some of that backstage story can be told, really.

A new or reinforced  why- building a platform that could add immense value through the use of data, none of the typical limiting constraints, an inspiring vision, a fine tuned learning machine, powered by real feedback, real needs, really hearing customers & msbi –still- strong and devoted community. And, not less amazing, brilliant , flawless execution. Incrementally, steadily, the pieces simply kept fitting together perfectly. How can these teams have such agility without never slightly compromising a stable, robust, and coherent platform down the road. I don’t know. It shouldn’t be possible. Am I dreaming?

Mission, vision, execution, brilliantly orchestrated.

To the unsung heroes, the amazing minds than could plan, design, envision such a brilliant architecture where pieces keep fitting magically together release after release… To the teams than then gradually brought it to life. Here’s to you. Know a few, but it’s pretty clear it’s only the tip of the iceberg. :)

I remember that right about Power BI GA back in ~July 2015, I was reading “Meaningful: The Story of Ideas That Fly” (truly inspiring book btw) and really felt I could be reading the story of Power BI there, the mindset, the why. Equally inspiring. heck. Even the book color reminded me of Power BI :)

From the book description…

”We don’t change the world by starting with our brilliant ideas, our dreams; we change the world by helping others to live their dreams. The story of ideas that fly is the story of the people who embrace them, love them, adopt them, care about them and share them. Successful ideas are the ones that become meaningful to others—helping them to see what’s possible for them.”

You see…? :)

Guess when you stop having customers and start having followers instead, amazing things are bound to happen.

Long post I, I warned right? Anyway, and also speaking of Azure, SQL Server,  Microsoft new support for Open Source platforms, R, Hadoop, Spark, so many others, such a powerful data platform we now have in our hands. Huge responsibility for us, data professionals, to take all this and deliver on the promise of immensely valuable data.

Should not surprise many, that I feel we’re not there yet (far from it, never been the one easily pleased with only gorgeous dashboards, sorry!). I believe we’re still capturing a tiny potential of the value hidden in all this data. But those thoughts will have to wait for another day. (hope it won’t take another year anyway…)  

Disclaimer: I don’t work for Microsoft, never have, although I work pretty much exclusively with Microsoft stack since I started my professional life, and also very sure, that I will stick with msft stack to the end of my work life. (that is, aside from my faithful Kindle and those precious things called moleskins )  

So, anyone still reading should have this in consideration. I may be biased… slightly…

Better late than never, Happy Birthday Power BI! :)

Rui

Read the full post here.

And celebrating the day :) also announcing that even if you can already use SmartCharts to push & sync data to the new Power BI Preview directly from Excel, you now have a dedicated app just to do that :) a brand new Send To Power BI Office App for Excel.

Give it a test drive and leave us your feedback or review. :) It will be greatly appreciated!

Download the new app from the App store.

For more info:

DevScope GitHub repository
Send To Power BI Excel App
SmartCharts Excel App
DevScope Office Apps
Rui Quintino Blog
Rui Romano Blog
www.devscope.net

Awesome day! :)

Rui

By now you know that there’s a new & strong developer story for the new Power BI,  and even if at the moment we have a simple API to test drive, it already allows to think of interesting scenarios. Particularly for real-time dashboards or  the click to insights pattern  I happen to obsess so much ….

Really, seems there will be no place for data to hide from Power BI :). When Power BI can’t get there… we will bring it to Power BI in a blink of an eye, in real time if needed, all using the brand new APIs.

Don’t have Power BI Designer? or even Excel? and need to quick load  some data to explore? Real time? Check out DevScope github  & use the awesome Power BI API Powershell modules to load your Data into PowerBI directly from the command line (for more info & a cool real time sample don’t miss  Rui Romano blog post).

Not forgetting we already have NuGet Power BI api modules for .Net, Xamarin,  iOS & Android by Sylvain Pontoreau .  Or more recently new sample web apps for Power BI api scenarios.

Adding that, as of… now :) you can also use SmartCharts to quick send & live sync to Power BI directly from Excel. :)

Start with your data as usual or the sample dataset and just click Send To Power BI.

Authenticate & authorize the app for Power BI access,  choose a Dataset and a table name :

and your data will be available in your Power BI account ready to more Power BI awesomeness

(note: if using date/time columns we also add some extra time related columns we happen to use in the app).

Your data ready to explore in Power BI:

Noting that you should be able to do all this on trial mode without any restrictions.  Got feedback and requests? let us know (there’s a feedback form in the app :) we will get back to you)

Get SmartCharts here, or checkout or other DevScope Labs  Office Apps here.

If you want to know more about the new APIs, some additional recommended reading:

• Power BI Developer Center
• Power BI Developer blog
• Power BI team github
• Richard diZerega  – Building Apps with the new Power BI APIs
• Power BI API adapter for Apache Spark
• Pushing Data into Power BI Preview Using the New REST API – Part 1
• Pushing Data into Power BI Preview Using the New REST API – Part 2
• DevScope’s awesome PowerShell module for Power BI API’s
• TechNet Wiki : PowerBI Api in .Net (en-US)

More to come :) stay tuned

Rui

note: sample AzureML experiments now published on the brand new AzureML gallery. Email Classifier-Training & Email Classifier – Scoring

Context: for years I’ve opted for an archive by folders strategy to handle my Inbox (not obsessed with inbox zero but  I try to keep it below ~50 inbox mails, and instead use inbox like a buffer – since my first contact with David Allen GTD years ago). true: once in a while I try to test the single  archive folder, but until now…always reverted back to my folders to organize so many parallel projects and threads going on

For this to work, I think one thing  is absolutely mandatory, very fast folder archiving and switching. Although have tried a few tools for this  I ended up coding some Outlook macros over the years to fit this… “peculiar” way of working and unproductive task switching…. 

So to archive a mail I just press alt-4, this window popups up, I type my search terms, enter, and it’s done. mail Archived

Or to switch context to a specific project or fast search, just press alt-3, search, enter and I’m there with the latest thread mails immediately available.

(that can admit, I obsess over searching…. “as you type” kind of search :) )

Fast forward to “the present”, #MachineLearning #DataIntelligence #AzureML era. I now have a few thousand “labeled” mails (to use machine learning terms) on several active folders. My macros were needing an improvement….  :)

So I exported my archive mails in the form:   |from | to | subject | time offset since fixed day|  format to a csv (tsv in fact)

and let AzureML do the heavy lifting of building my very personal email classifier suggestion web service, putting some multiclass classification models & also text handling /feature hashing AzureML features to work in a more useful scenario than classifying flowers :)

Sticking with the neural network model for now (default params), created and adjusted the scoring experiment & published the web service.

Used the VBA code sample from the new cool AzureML generated Excel files,

few lines of code added on my Outlook macros, based on AzureML Excel VBA code/macros

and we get a pretty impressive auto classifier ready to use & help manage our inbox, suggesting the folders where the message belongs when archiving.(note: also triggered when sending messages, archiving both sent and original message if needed.)

So for example,  if I’m disturbing Joana with another annoying mail :) about SmartCharts new features AzureML advises me:

On the other hand, if it were an help request to Romano on some stream analytics samples AzureML would opt for:

press Enter, for now it’s needed…:) ,  and that’s done. How cool is that? :)

(have to say that the accuracy is not 100% obviously , but pretty damn useful already)

All this running on a free AzureML workspace. Up & running & minutes, from training to online web service

(excluding the time to slightly adapt AzureML VBA code to call the web service & get my mail properly exported)

Training & scoring experiments are available in AzureML gallery fo you to test drive. Off course I loaded a small sample of my mail in those, so you will have to load your own to really see how it works.

…now, to be close to perfection this would benefit from automated data update & model retraining, and that will be the case for use the new AzureML training APIs.

But that will have to wait! :) (true, reduced some time processing my inbox, but not that much…!)

btw- hope to get feedback on this scenario, ex: feature engineering tips and model tuning suggestions to improve the model results

Cheers

Rui