Clifton Phua's Blog

The Good Taxi Driver's Final Shift

2016-09-23T23:44:00.001+08:00

I told the taxi driver my destination. He paused briefly with his elbow on the wound-down car window and said, "Ok".

After hopping on the back seat, he started driving and explained. He did not have sufficient money in his cash card and the journey did not require passing by any ERP gantry. Like almost all taxi drivers I encounter at 7-ish in the morning, he commented on the long journey in my daily commute to work.

He is a big, burly man who wore sunshade clip-ons on his spectacles.

"Today is my last day driving taxi. I driving for 23 years now. Have to do heart operation at 2:30pm today and specialist wrote letter for LTA. I cannot drive a taxi anymore.", he sighed.

He caught me by surprise and I stared at a black Buddhist amulet hanging from the car's rear view mirror. "I am sorry to hear that. Do you have children?"

As the taxi pulled up to a complete stop at the traffic light, he answered. "Uncle is 60. My younger one is 6 and I just dropped off my 13 year old son in school and told him to study hard when Papa is not around. Don't be like me. My son cried.", a few seconds later, he seemed to have taken comfort in saying that his children are smart.

I was getting concerned. "Do you have medical insurance?"

"No. And this is a big mistake not to have any. This is going to be my 2nd operation and it is going to cost $30,000. My 1st heart operation was 3 years ago, and I could not work for a year and had to depend on government financial assistance and help from close friends. This time it is more serious.

You are my 4th passenger, as I have started work at 4am. After dropping off my son, I had some breakfast at 7am. Have to fast several hours before the operation."

By then, I realized that this taxi driver is in one of the darkest moments of his life. There is high chance that he is going to die soon. In the event that he survives, he cannot provide for his family during or after his recovery from this impending surgery. Right now, he is still driving his taxi. Maybe he needed to talk to someone. Anyone.

I continued to listen intently. We don't hear stories like this everyday. Certainly not directly from the dying.

"My sickness is connected to me working too hard in my younger days. In the day, Uncle work as a chemical technician and part-time as taxi driver. Since my son was born, I began to drive taxi full-time to have more flexibility in looking after the family. Also, I was ignoring symptoms like numbness in the left side of my body...", he explained slowly. "Don't be like me."

"I didn't expect to get married at all. My recently-deceased mother always insisted that I had to. It was by coincidence that I met my wife, late in life, as I was helping out a friend with his transport business... My wife did not have to work after we got married, and now she has to work as a helper at the child care centre because of this..."

Along the journey, the taxi driver had a few hands-free conversations in Cantonese. His friends have called to wish him well, one of them reprimanded him for working right now.

"These are my close friends from volunteering at the Bukit Merah old folk's home in Singapore.", and he went on to talk about his volunteer work in orphans' homes in Thailand when he was younger. He could speak fluent Thai and said that I should volunteer too. I smiled.

The taxi driver said that he loved his 7 year old taxi, and as he said that, he stroked the steering wheel. He described the study room within his HDB flat as his achievement. He has accumulated a collection of books with a comfortable chair and foot massager machine.

"我不喝酒不吸烟，经常锻炼身体, 很难接受有这个病。"

At a traffic light stop, he showed the text messages he got from his taxi company, praising him for help and kindness to many passengers. He showed me the huge scar from the 1st heart operation which still gives him much physical pain as his some of his ribs had to be cut off.

As a Buddhist, he said he found acceptance. He believes that in his previous life, he must have done a lot of bad things and even though he had tried to do a lot of good in this life, it is not good enough to make full amends for his past life misdeeds.

"For good luck", I leaned forward and whispered, handing over a small monetary token after reaching my destination. He put his palms together as a sign of thanks.

Four Worthy (Data Science) Blog Links from August 2016

2016-08-31T23:19:00.000+08:00

Mosaic version of Chijmes Hall

Google open-sources Parsey’s Cousins, a set of parsers for 40 more languages - from VentureBeat
"Parsing language might not sound like a big deal — it involves looking at a sentence and picking out the nouns, verbs, adjectives, and so on. But Parsey McParseface works at Google scale, that is to say it is very good, good enough for machines to use it to understand users’ web search queries. Researchers can now take advantage of the technology in more languages without worrying about where they’ll get the data for teaching the models."

Wonder is a bot that will remember anything for you - from TechCrunch
"Basically, you just text Wonder the information you’ll need to recall at a later date, and it stores that for you in its system. When you’re trying to later remember something, you just text Wonder a question, like “Who’s our company’s dental insurance provider?,” “When’s the next company meeting?,” or whatever other information you’ve previously fed into it by way of text message. The bot will promptly respond with the answer."

62% of Enterprises will Use AI by 2018 - from What's The Big Data?
"Keep in mind that 'AI technologies' is a broad term that includes machine and deep learning, recommendation engines, predictive and prescriptive analytics, automated written reporting and communications, and voice recognition and response."

Here’s how Baidu’s founder thinks AI will change the world - from TechInAsia
"On the second question, what industries will AI influence, that’s something we’re constantly thinking about. I think it will influence many industries. But the biggest industry it will influence is China’s biggest industry in general: manufacturing."

Nine Worthy (Data Science) Blog Links from July 2016

2016-07-24T16:00:00.002+08:00

Generated from Deep Dream on a Flower Basin from Intercontinental Danang

A Future for R: Slides from useR 2016 - from R-Bloggers
"At the end of the second day, I presented A Future for R (18 min talk; slides below) on how you can use the future package for asynchronous (parallel and distributed) processing using a single unified API regardless of what backend you have available, e.g. multicore, multisession, ad hoc cluster, and job schedulers."

Three Impactful Machine Learning Topics at ICML 2016 - from KDNuggets
"Three of the tutorial sessions I attended were quite impactful. Anyone working on conversational apps, chatbots, and deep learning would be interested in these topics.

Deep Residual Networks: Deep Learning Gets Way Deeper by Kaiming He
Memory Networks for Language Understanding by Jason Weston
Recent Advances in Non-Convex Optimization and its Implications to Learning by Anima Anandkumar"

Storytelling: The Power to Influence in Data Science - from KDNuggets
"Six Ways to Improve Your Storytelling Skills:

Read as much as possible
Write as much as possible
Find your voice
Visualizations First
Listen to Stand-up Comedy
Know your Audience"

Machine Learning Applications by Industry - from What's The Big Data?
"Manufacturing, Retail, Healthcare and Life Sciences, Travel and Hospitality, Financial Services; Energy, Feedstock, and Utilities"

Approaching (Almost) Any Machine Learning Problem | Abhishek Thakur - from No Free Hunch
"This post focuses on the second part, i.e., applying machine learning models, including the preprocessing steps. The pipelines discussed in this post come as a result of over a hundred machine learning competitions that I’ve taken part in. It must be noted that the discussion here is very general but very useful and there can also be very complicated methods which exist and are practised by professionals.

We will be using python!"

Bot thots - from O'Reilly
"Part of the reason we’re excited about bots is that the app economy has stagnated: “the 20 most successful developers grab nearly half of all revenues on Apple’s app store,” notes The Economist. It’s hard for users to discover new apps from among the millions that already exist, and the app-installation process involves considerable friction. So, the reasoning goes, bots will be great because they offer a way to skip the stagnant app stores and offer a smoother “installation” process that’s as simple as messaging a new contact."

How to write good tests in R - R-Bloggers
"In general, tests should be:

self contained – don’t rely on code outside of the test block;
isolated – independent and not commingled with other test cases;
unique – focus on a unique behavior or property of a function that has not been previously tested;
useful – focus on edge cases and inputs where the function may behave erratically or wildly;
scalable – easy to write variations without a lot of ceremony."

The Big Data Ecosystem is Too Damn Big - from KDNuggets

"Always Start With a Use Case

Don’t get sold by shiny tech. In a recent Gartner survey, by far the top big data challenge cited by respondents was “determining how to get value from big data” (58% of respondents). How do you remedy that? Always start with defining your use case, then work your way toward finding the technology that will support it.

Consider Control vs. Democratization

As hinted at above, it may be tempting to give yourself/your team fine-level controls with tools that allow you to code. But be wary of how much control you actually need – is the greater good better served by getting data into the hands of more people in the organization with self-service tooling? Search for the right balance.

Think Future-Ready

We’ve already seen it. The industry is contracting, expanding, contracting, expanding. That’s why it’s incredibly important that as you evaluate your technology purchase, you look for signs the technology itself is “future-proof” or “future-ready” through modular, “pluggable” architecture. Because, while you may not want toleap on the next shiny new project or standard, you’ll want the option to migrate to it as it becomes prudent to do so."

Objections vs. excuses - from Seth Godin
"Objections are healthy. When someone is being offered a new opportunity or product, it's not unusual for there to be objections.
...
Excuses, on the other hand, are merely fear out loud."

Four Worthy Blog Links from June 2016

2016-06-30T00:30:00.000+08:00

Big Data in Public Transportation - from HortonWorks
"Big Data is transforming both the Plan phase and Operations phase of the Rail transportation. City authorities who are in-charge of this transportation service, should consider leveraging Big Data for 4 key use cases"

What are the Best Machine Learning Packages in R? - from R-Bloggers
"In this article we will be more focused on packages used in the field of Machine Learning."

I tried a ton of chatbots. Here’s the good and the bad. - from TechInAsia
"Telegram takes it up a notch with inline bots, which brings bots into existing conversations...

They’re nice features of existing web services, but not full-fledged businesses. Facebook and Telegram don’t even have official bot shops yet. Here’s a good unofficial bot store.

For bots to replace apps (and I’m not sure if that will ever happen, just like how radio still exists today), they’ll need to reach new heights in AI and usability. That could take years."

"So busy doing my job, I can't get any work done" - from Seth Godin
"The work, on the other hand, is the thing you do that creates value. This value you create, the thing you do like no one else can do, is the real reason we need you to be here, with us.

When you discover that the job is in the way of the work, consider changing your job enough that you can go back to creating value.

Anything less is hiding."

Eight Worthy Blog Links from May 2016

2016-05-29T23:06:00.000+08:00

This Guy Created an Awesome Miniature ‘Jurassic Park’ For His Tortoise - from 22Words

Bias built in to crime prediction - from FlowingData
"ProPublica investigates the reliability of these scores, using data from Broward County, Flordia, between 2013 and 2014"

Startups Mining New Data Sources for Investors - from What's the Big Data?
"Satellite Data... Alternative Credit... Local Prices"

Building Data Systems: What Do You Need? - from Silicon Valley Data Science
"There are a lot of capabilities to think about, and it can seem daunting. Remember that at the highest level, you are aiming to give your teams as much visibility as possible into what’s happening through instrumentation, implementing processes that provide validation at every step, and automating the tasks that make sense."

Uber single-handedly driving up COE prices in Singapore - from TechInAsia
"They should not be in the private cars category. The Taxis are not, don’t see why Uber or Grab should be."

Making data analytics easier for data scientists, developers, and DevOps - from O'Reilly
"The open source Trusted Analytics Platform (TAP) eliminates those silos in a way that serves data scientists, application developers, and DevOps engineers—individually and collectively."

Google open-sources SyntaxNet, a natural-language understanding library for TensorFlow - from VentureBeat
"The parser, which goes by the name Parsey McParseface and can automatically figure out whether a word is a noun or a verb or an adjective just like your third-grade English teacher, is the most accurate one in the world, Google says, beating out its own technology."

Leicester City’s Stunning Rise, In Two Charts - from FiveThirtyEight
"Manchester United has spent more money on new players in the last two years than Leicester has spent in the 132 years it’s existed."

The "No Toxic People" or "No Asshole" Rule at the Workplace

2016-04-30T23:30:00.000+08:00

Disclaimer: Not referring to any specific entity, just highlighting this in an abstract manner because this is an issue at every workplace.

There is much studied and written about this topic, such as 10 Toxic People You Should Avoid At All Costs and The No Asshole Rule.

Toxic people can be within the organization, such as bosses, peers, or subordinates; can be outside the organization, such as customers, potential customers, partners, governments, or vendors. When one encounters or deals with them, it is unlikely to forget them as their negativity hits hard. One can control reactions towards them, yet this is very difficult to do.

Maybe toxic people, are the way they are, due to their inherent nature, previous experiences, or specific circumstances. Maybe they can evolve into their current form as a result of their organizational culture or comes from top down.

Some direct long-term consequences of having or dealing with toxic people include:

Morale issues
Employee turnover
Hiring of more toxic people
Bad organizational reputation

Good leadership has to adopt a zero tolerance policy towards toxic people, and this includes having the ability to detect them quickly and prevent them from causing further damage.

Seven Worthy Blog Links from April 2016

2016-04-30T15:00:00.000+08:00

Swapping faces with statues is rather disturbing - from BoingBoing

Interconnectedness of the galaxies - from FlowingData

5 Origins of Data Science - from What's the Big Data?

Smart cities market to double to $1.4 trillion by 2020 - from ReadWrite

Facebook will announce chatbot and live chat APIs at F8 - from TechCrunch

Chinese villagers loot UNESCO World Heritage site while it's being cleaned - from Mashable

Woman has a meltdown while playing terrifying VR game - from Mashable

Some Animal Pictures from Taipei Zoo

2016-04-03T15:00:00.000+08:00

A Few Panoramic Pictures From Taiwan

2016-04-03T09:00:00.000+08:00

Flying Cow Ranch Panorama 1

Flying Cow Ranch Panorama 2

Maokong Station Panorama

Taipei Children's Amusement Park Panorama

View from Jiufen Outdoors Panoroma

Yehliu Panorama 1

Yehliu Panorama 2

Seven Worthy Blog Links from March 2016

2016-03-31T23:30:00.000+08:00

Advanced Analytics Software’s Most Important Feature? Gartner Says it’s VCF - from R-Bloggers
"The watershed change between these two plots is hinted at by two companies that are missing in the more recent one: Salford Systems and Tibco. The important thing is why they’re missing. Gartner excluded them this year, “…due to not satisfying the [new] visual composition framework [VCF] inclusion criteria.” VCF is the term they’re using to describe the workflow (also called streams or flowcharts) style of Graphical User Interface (GUI). To be included in the 2016 plot, companies must have offered software that uses the workflow GUI. What Garter is saying is, in essence, advanced analytics software that does not use the workflow interface is not worth following!"

Exploring the Intersection of Art and Machine Intelligence - from Google Research Blog
"In addition to helping us gain a deeper understanding of how NNs worked, these techniques also produced strange, wonderful and oddly compelling images."

Five Business Challenges Data Can Solve - from Silicon Valley Data Science
"

Infrastructure is inhibiting growth
Infrastructure is inhibiting development
Undergoing an analog to digital transformation
Changing business models
Unifying fragmented offerings

"

The SkyWall 100 bazooka captures drones with a giant net - from TechCrunch

Smart City Projects Rumbling to Life Around the World - from ReadWrite
"They identified 235 projects around the world. Roughly 85% of the projects are focused on issues related to energy, transportation, or government. That’s up from 170 projects in the third quarter of 2013 when the firm first began looking at the smart city space."

This Fluffy White Bunny Eating Strawberries and Cherries Is Like Something Out of a Horror Movie - from 22 Words

We Now Have Algorithms To Predict Police Misconduct - from FiveThirtyEight
"These researchers, part of the White House’s Police Data Initiative, say their algorithm can foresee adverse interactions between officers and civilians, ranging from impolite traffic stops to fatal shootings."

Six Worthy Blog Links from February 2016

2016-02-29T23:30:00.000+08:00

International impact of China’s economic slowdown - from FlowingData
"China’s economic slowdown means a major decline in imports from other countries, which leads to significant effects in these areas."

Policing the Future - from The Marshall Project
"HunchLab, produced by Philadelphia-based startup Azavea, represents the newest iteration of predictive policing, a method of analyzing crime data and identifying patterns that may repeat into the future. HunchLab primarily surveys past crimes, but also digs into dozens of other factors like population density; census data; the locations of bars, churches, schools, and transportation hubs; schedules for home games — even moon phases. Some of the correlations it uncovers are obvious, like less crime on cold days. Others are more mysterious: rates of aggravated assault in Chicago have decreased on windier days, while cars in Philadelphia were stolen more often when parked near schools."

Salesforce acquires machine learning startup PredictionIO - from VentureBeat
"The PredictionIO open source software will remain available under an open source Apache license. And the PredictionIO cluster service on top of the Amazon Web Services CloudFormation service will become available for free, Chan wrote. PredictionIO offers an enterprise service tier, but pricing information isn’t listed on the startup’s website."

Top 10 Data Visualization Projects on Github - from KDNuggets

The Decline of Amazon Mechanical Turk - from A Computer Scientist in a Business School
"Overall, we observe a YoY decline of almost 50% in terms of active requesters.

What is driving the decline? Hard to tell. Perhaps requesters abandon crowdsourcing in favor of more automated solutions, such as deep learning. Perhaps requesters with long running jobs build their own workforce (eg using UpWork). Perhaps they use alternative platforms, such as Crowdflower. Or perhaps my own metric is flawed, and I need to revise it.

But, unless we have a bug in the code, the future does not seem promising for Mechanical Turk. And this is a shame."

Evolution of the Netflix Data Pipeline - from Netflix Blog
"There are three major components:

Data Ingestion - There are two ways for applications to ingest data.

use our Java library and write to Kafka directly.
send to an HTTP proxy which then writes to Kafka.

Data Buffering - Kafka serves as the replicated persistent message queue. It also helps absorb temporary outages from downstream sinks.
Data Routing - The routing service is responsible for moving data from fronting Kafka to various sinks: S3, Elasticsearch, and secondary Kafka."

Six Worthy Blog Links from January 2016

2016-01-31T23:30:00.000+08:00

Cheryl Yeoh on her MaGIC legacy - from TechInAsia
"What are key lessons you learned while heading MaGIC?"

Living Off Hackathons: The Possible Rise Of The Pro-Hacker - from TechCrunch
"There is now a compelling argument for ambitious programmers to live off hackathons: They can forget the long-term slog of building a startup and instead focus on the fun part — the first few days of innovation."

ISIS Has Its Own Encrypted Chat App - from TechCrunch
"Alrawi can’t be downloaded from Google Play. Instead it must be installed from shady back alleys of the Internet."

How to Create an Award-Winning Prezi: Q&A with Prezi Awards Winner Anna Bojti - from Prezi Blog
"I always like to tell my clients that good Prezi design is not about zooming and rotating. The most important thing is to have a clear structure, well-defined messages, and, of course, good graphic design. And a good story always makes the presentation unforgettable for the audience."

Tearful Obama calls for 'sense of urgency' to fight gun violence in America - from BoingBoing

Analyze Social Media Data in Real Time - from Big Data University
"…using Spark Streaming to analyze social data. I have created a streaming program that is constantly running, fetching Twitter data in real-time, and also clustering the tweets based on their text and location, using the k-means algorithm…"

Wikipedia Mining reveals hidden Revolution of Human Priorities - from KDNuggets
"Wikipedia data mining may reveal changes over time in the human perception of the world, and may also serve as an independent reliable quantitative method of investigation of historical events."

Four Worthy Blog Links from December 2015

2016-01-09T23:48:00.000+08:00

Ed Sheeran played this $2-a-ticket show, but no one wanted to see him - from Mashable
"It took them over two hours to convince anyone to pay the money to see Sheeran, but after the first couple spread the word business did pick up a bit."

Data Analytics in 2016- The 6 Trends We Must Be Aware Of - from Teradata International Blog
"The big question on every executive’s lips next year will be how to “monetize” their data? How will we leverage both the technology and the people resources of the organisation to derive value from the data that we generate and consume? Monetization of the data can be either in the form of increasing revenue streams within the organisation, finding new revenue streams or using the data to create efficiencies within the organisation to reduce cost."

Police violence incident data - from FlowingData
"dataset for download for shootings from 2013 to present."

Which Cities Share The Most Crime Data? - from FiveThirtyEight
"New Orleans is arguably the queen of open police data, at least in terms of the quantity of information it provides. New Orleans has placed its entire calls for service system online since 2011, offering the public a significant opportunity to evaluate its police force. A recent analysis of the data by several local media outlets found that the New Orleans Police Department’s response times had increased dramatically. Longer response times had led to fewer investigations, which in turn had deflated crime statistics by around 6 percent. After the news reports, the police department changed its policy regarding how crimes were reported."

Six Worthy Blog Links from November 2015

2016-01-09T23:46:00.000+08:00

Plotting Russian AiRstRikes in SyRia - from R-Bloggers
"http://russia-strikes-syria.silk.co/
The data includes the latitude and longitude of the strikes along with other background information."

Back from the dead: 6 species we thought were extinct, but really aren't - from Mashable
"Lazarus species are extremely rare, and sadly we are certain that many more species have indeed gone extinct in the last 100 years."

Cat vs. cucumbers - from BoingBoing

49 of the Most Powerful Pictures You’ve Ever Seen - from 22Words

Intelligent Transportation Systems for Midsized Cities - from SmartDataCollective
"When intelligent transportation systems take advantage of advanced data analytics, the value that results includes optimized infrastructure management and operations, as well as improved transportation services in terms of variety of modes, safety, reliability, and efficiency. Transportation data analytics also contribute greatly to real-time improvements to traveler / user experiences."

Analyzing 1.1 Billion NYC Taxi and Uber Trips, with a Vengeance - from R-Bloggers
"How bad is the rush hour traffic from Midtown to JFK? Where does the Bridge and Tunnel crowd hang out on Saturday nights? What time do investment bankers get to work? How has Uber changed the landscape for taxis?"

Four Worthy Blog Links from October 2015

2016-01-09T23:43:00.000+08:00

Colonial-era church emerges from the water in a reservoir in Mexico - from Mashable
"The church in the Quechula locality was built by a group of monks headed by Friar Bartolome de la Casas, who arrived in the region inhabited by the Zoque people in the mid-16th century."

A $470 Billion Reason Insurers Are Turning to Data Solutions - from SmartDataCollective
"While most insurance companies realize they need to adapt to consumers’ expectations for smooth, omnichannel experiences, almost half admit that they don’t yet have a realistic plan in place."

Text Mining with R - from R-Bloggers

The Data Science Machine, or ‘How To Engineer Feature Engineering’ - from KDNuggets
"To measure the DSM's performance, it was entered in competitions at KDD Cup 2014, IJCAI, and KDD Cup 2015, where, as mentioned, it outperformed more than 2/3 of the human competitors."

One Worthy Blog Link from September 2015

2016-01-09T23:42:00.000+08:00

Knight News Challenge on Data - from KDNuggets
"Winners will receive a share of $3 million"

Five Worthy Blog Links from August 2015

2016-01-09T23:39:00.000+08:00

Extract data from PDF files and export to CSV - from FlowingData
"The user interface got an overhaul and it's now easier to grab data from multiple pages."

Significant Data Breaches – July 2015 - from Ikanow
"a membership of 37M"

Streamgraphs in R - from Revolutions
"connect categories together into continuous polygons, and can even be aligned to the middle so that no one category gets favoured status"

How Much Data is Generated Every Minute? - from What's The Big Data?

Constructing a network of politicians from newspaper data - from R-Bloggers
"What do we find? Rather unsurprisingly, Angela Merkel as well as her 'most important' ministers Sigmar Gabriel (economy and energy), Frank-Walter Steinmeier (foreign affairs), and Wolfgang Schäuble (finance) are mentioned most often in the articles and hold strong connections to the chancellor."

Six Worthy Blog Links from July 2015

2015-08-02T10:51:00.000+08:00

How Airbnb built a data science team - from SimplyStatistics
"We needed to find a way to democratize our work, broadening from individual interactions, to empowering teams, the company, and even our community."

Predictive Policing – Free Book - from KDNuggets
"The Book on Predictive Policing –Taking a chance for a safer society (free ebook) is an easy to digest introduction to the world of predictive analytics and Big Data. The book is written from a policing perspective and shows interesting views in how the power of the police force can be increased by focusing on predictive policing."

The War On Terrorists’ Tweets - from DefenseOne
"This proposal connects to efforts to understand terrorist use of social media, its national security implications, and ways to counter it."

A surveillance system that watches over an entire city - from FlowingData
"In 2004, when casualties in Iraq were rising due to roadside bombs, Ross McNutt and his team came up with an idea. With a small plane and a 44 mega-pixel camera, they figured out how to watch an entire city all at once, all day long. Whenever a bomb detonated, they could zoom onto that spot and then, because this eye in the sky had been there all along, they could scroll back in time and see - literally see - who planted it."

R financial time series tips everyone should know about - from R-Bloggers
"… introduce a list of the most useful tricks I came across when dealing with financial time series in R. Some of the functions presented here are incredibly powerful but unfortunately buried in the documentation hence my desire to create a dedicated post. I only address daily or lower frequency times series."

Introducing the nominatim geocoding package - from R-Bloggers
"Just like Google Maps, these services are not meant to be your freebie-access to mega-bulk-geocoding. You can and should pay for that.

But, when you need a few items geocoded (or want to lookup some interesting things on OSM since it provides special phrases to work with), Nominatim lookups can be just what’s needed."

Five Worthy Blog Links from June 2015

2015-08-02T10:26:00.000+08:00

Network analysis with igraph - from R-bloggers
"This is the part that I loved most: With just two packages, I was able to make not-so-ugly charts, saying publishable."

Aggregate the Los Angeles realtime bus feed - from FlowingData

Here comes the industrial Internet — and enormous amounts of data - from VentureBeat
"If companies are going to make use of all this data they’re generating, they’ll need to start pulling them together. That’s where big-data analysis tools like Apache Spark, data-management and storage tools like Hadoop, and search engines like Maana and Elasticsearch come in."

Southeast Asian walking fish lives up to its name - from Mashable
"The climbing perch, which scientists warn could be wiggling its way to Australia from Papua New Guinea, has a unique ambulatory style. It uses sharp, spindly gills to pull itself across dry land, and can live out of the water for up to six days."

This Man Went Kayak Fishing But NEVER Expected To Reel In THIS… - from 22Words

Seven Worthy Blog Links from May 2015

2015-05-31T00:11:00.000+08:00

More Than Just a Title: How to Identify a Data Scientist - from SmartDataCollective
"Strong technical chops alone do not make an exceptional data scientist, so when recruiting we look for a combination of technical and non-technical skills."

21 Essential Data Visualization Tools - from KDNuggets
"Of course, they have both strong and weak sides, since there is no one perfect tool to visualize the metrics. All I can do is to recommend you trying them yourself and combining them in order to maximize the efficiency of visualizing data."

SparkR preview in Rstudio - from R-Bloggers
"Apache Spark is the hip new technology on the block. It allows you to write scripts in a functional style and the technology behind it will allow you to run iterative tasks very quickly on a cluster of machines. It’s benchmarked to be quicker than hadoop for most machine learning use cases (by a factor between 10-100) and soon Spark will also have support for the R language. As of April 2015, SparkR has been merged into Apache Spark and is shipping with a new version in an upcoming release (1.4) due early summer 2015. In the meanwhile, you can use this tutorial to go ahead and get familiar with the current version of SparkR."

Why the creators of the Julia programming language just launched a startup - from VentureBeat
"And Julia is a big deal — it’s a free alternative to proprietary tools for doing data science, like MathWorks’ MATLAB and Wolfram’s Mathematica, and it’s more contemporary than open-source languages R and Python."

Why Windows 10 is the 'last' version of Windows - from Mashable
"Switching to the Windows-as-a-service model is a significant change to Microsoft's business model, even more so because Windows 10 will be a free upgrade to anyone running Windows 7 or later."

Hourglass full of magnetized "sand" - from BoingBoing
"This $14, 100-second hourglass is filled with magnetized iron filings that form beautiful fans and other shapes as the sand drains into the bottom bulb."

This Is Perhaps the Most Impressive 5 Seconds You’ll EVER See - from 22Words

Seven Worthy Blog Links from April 2015

2015-05-03T21:41:00.000+08:00

10 captivating short stories everyone should read - from BoingBoing
"Johnny has a great list of short stories from Asimov, Hemingway, King, Vonnegut, Gaiman, and others, with links to the stories."

How can teachers of (large) online classes use text data from online learners? - from Statistical Modeling, Causal Inference, and Social Science
"The instructor could run this program, look at the results, see what makes sense, and see what doesn’t make sense. Ideally the program would come with some feedback options so that Reich et al., as developers of the software, can improve the model and make it more useful. Thus, a system with its own built-in mechanism for improvement."

Which analytic technology problems are important to solve for whom? - from DBMS2
"Bottom line: Almost every interesting analytic technology problem is worth solving for some market, but please be careful about finding the right match."

Mapping Flows in R … with data.table and lattice - from R-bloggers
https://s3.amazonaws.com/assets.datacamp.com/img/blog/data+table+cheat+sheet.pdf

IBM Opens Up Its Threat Data as Part of New Security Intelligence Sharing Platform - from SmartDataCollective
"… opened up over 700 terabytes of data about vulnerabilities, attacks and other threats through a new cloud-based threat intelligence sharing platform called IBM X-Force Exchange… "

A SIEM is not enough: moving to a complete cyber security solution - from Ikanow
"However, data collection alone isn’t enough. Today’s organizations need threat analytics to transform data into actionable intelligence."

Text Mining with Impala - from Cloudera Blog
"Our extensive evaluations show that INDREX can return results for most text mining queries within a few seconds in Impala. Most operations on text data are embarrassingly parallel, such as local and join operations on a single sentence or operations on a single document."

Seven Worthy Blog Links from March 2015

2015-03-28T23:00:00.000+08:00

Paul Allen discovers wreckage of major WWII Japanese battleship - from Mashable
"Microsoft cofounder Paul Allen says he's found the wreckage of one of Japan's largest battleships in the Sibuyan Sea off the Philippines after an eight-year long search for the vessel."

Lockheed Martin's laser weapon destroyed a truck from a mile away within seconds - from Mashable
"In layman's terms: It showed how laser weapons can be used for counterterrorism and to protect the military."

Cartoon: US Chief Data Scientist Most Difficult Challenge

SQL-like Query Language for Real-time Streaming Analytics

Traffic gridlock simulation

htmlwidgets: Create interactive web charts in R

Using PageRank to find Anomalies in healthcare

Six Worthy Blog Links from February 2015

2015-03-08T12:21:00.000+08:00

Two Most Important Trends in Analytics and Big Data in 2015 - from KDNuggets
"To give a better idea of impact of how developing countries will benefit from big data and analytics, to “improve” their processes and operations for the welfare of the public, here are just a few examples:
•Predicting better where the economy will head with more specific data, what will happen if any of the factors were to change
•Trending crime to control and prevent the spread or increase, and eventually reduce
•Tracking migration of micro-societies for their health and well-being e.g. indigenous"

6 tips for managing highly intelligent employees - from Mashable
"Don't manage: Guide. IQ and experience are two attributes you want your team members to over-index on, but there are plenty of other attributes that also matter, including judgment, work ethic, communication skills and teamwork. Unlike raw IQ, these are things that can be developed in any employee. Your job is to provide individualized guidance to each of your team members in the areas that they need to develop."

Who’s Using Cyberthreat Intelligence and How? - from SANS
"The idea behind cyberthreat intelligence is to provide the ability to recognize and act upon indicators of attack and compromise scenarios in a timely manner. While bits of information about attacks abound, cyberthreat intelligence (CTI) recognizes indicators of attacks as they progress, in essence putting these pieces together with shared knowledge about attack methods and processes."

Big Data Failures Owe More To Business Culture Than Technology - from ReadWrite
"There are many factors that go into the making of a successful Big Data implementation. However, the single biggest factor that we observed was that organizations that have a strong operating model stood apart. This operating model has multiple distinct elements, which include, among others, a well-defined organizational structure, systematic implementation plan, and strong leadership support."

Uncertainty over why rabbit prefers to walk on its front legs - from BoingBoing
"This bunny was one of many purchased by a Chinese zoo as live food for some of the animals it keeps.

No one at the zoo knows why the bunny walks like this, but I do: a little female spider taught the bunny how to walk on its front legs, making it a national sensation and sparing it from the jaws of a giant python."

Marnie the Dog knows how to find a good deal - from Mashable

Five Worthy Blog Links from January 2015

2015-01-31T22:35:00.001+08:00

Attack Attribution in Cyberspace - from Schneier on Security
"The only way out of this, with respect to the Sony hack and any other incident of cyber-aggression in which we're expected to support retaliatory action, is for the government to be much more forthcoming about its evidence. The secrecy of the NSA's sources and methods is going to have to take a backseat to the public's right to know. And in cyberspace, we're going to have to accept the uncomfortable fact that there's a lot we don't know."

To-do lists are evil. Schedule everything. - from Boing Boing
"Scheduling forces you to confront the reality of how much time you actually have and how long things will take. Now that you look at the whole picture you're able to get something productive out of every free hour you have in your workday. You not only squeeze more work in but you're able to put work into places where you can do it best."

Facebook Open-Sources Some Of Its Deep-Learning Tools - from TechCrunch
"Facebook, too, has done quite a bit of work in the area and today, the company is open-sourcing some of its projects around the Torch7 computing framework for machine learning. Torch has long been at the center of many machine learning and artificial intelligence projects in academic labs and at companies like Google, Twitter and Intel."

How To Hire A Data Scientist - from ReadWrite
"Hence, while it's critical to ask questions like "Can you give me an example of when you’ve developed an algorithm from a framework/research paper?” so as to tease out whether they have the blend of math/statistical modeling and programming skills required of a good data scientist, it's equally important to ask things like, "Tell me about a time when you’ve improved a business process?"

They need, in other words, to understand how the data impacts the business. Find that sort of data scientist and you're golden."

Building and deploying large-scale machine learning pipelines - from O'Reilly Radar
"As algorithms become even more pervasive, we need better tools for building complex yet robust and stable machine learning systems. While other systems like scikit-learn and GraphLab support pipelines, a popular distributed framework like Apache Spark takes these ideas to extremely large data sets and a wider audience. Early results look promising: AMPLab researchers have built large-scale pipelines that match some state-of-the-art results in vision, speech, and text processing."

Six Worthy Blog Links from December 2014

2014-12-31T22:47:00.000+08:00

15 tips on turning your finances around in 2015 - from Mashable
"…there's no standard advice about personal finance except for the overarching one: Spend less than you make…"

If programming languages were vehicles, what would be R, Python, SAS, and SQL? - from KDNuggets
"I can think of R as a flying car - very powerful, but only experts can handle it.

SQL, born in 1970s, would be a classic 1970-s Ford Mustang. No modern gizmos, but still fast and reliable, if properly maintained.

SAS may be a corporate luxury car - perhaps a Lincoln Continental or Mercedes. Great car, but expensive."

What to expect from Apple in 2015 - from Mashable
"Apple, I believe, will eventually become one of the most important companies in healthcare," said Bajarin, who thinks the company’s desire to help people track and manage their health digitally goes all the way back to the late Steve Jobs. "Apple’s role in health is Steve's last great gift to mankind. I believe, it all came out of his illness. There has to be a better way to monitor one's health and automate the interaction with the healthcare system."

These Were The Top 10 Most Popular Searches On Google In 2014 - from TechCrunch
"Google notes that 2048 and Flappy Bird, both one-man projects, beat out Destiny, the most expensive game ever developed."

37K sentiment-analysis words associated with emotion scores - from BoingBoing
"emotion analysis represents a natural evolution of sentiment analysis"

Computational power and cognitive augmentation - from O'Reilly Radar
"Tomorrow’s interfaces won’t be about mobility, or haptics, or augmented reality (AR), or HUDs, or voice activation. I mean, they will be, but that’s just the icing. They’ll be about interruption.
...
That interruption will take many forms — a voice on a phone, a buzz on a bike handlebar, a heads-up display over actual heads. But behind it is a tremendous amount of context that helps us to decide better."