Aspie Econ

On Being a Data Scientist-- the Elliptic Curve Axiom

2013-05-20T20:11:00.001-07:00

May 20th, 2013 (MVCA)
I derived Elliptic Curves over K in honor of the Tableau IPO or as I call it:
"Elliptic Curves are everywhere, so pay up fool!!!!"

But, don't fear for I've cracked the code and now I present all of 'em to you, free of charge:
"Derivatives Traders be damned!!!!!!"

Streets walled in hedges
Noting that I am setting up [CarlSagan^50] for ellipt-eration by careful and loving attention to [Boob-raider to N = 641K] and any issues with performance would be directly attributed to size of the set, which of course, poses a problem against 'scale' and 'efficiency'. To remedy, I'd loop some data scientry logic and then wait patiently to receive the inevitable call from US-HLS.
"Those bastards."

Tales from the Cryptography
# Data = Boob-raider to N = 641K
# Math = Relative-Cryptography or [ModP*Dank Rocket Scientry = a New Hope]
# P, 2P, 3P and 4P (more to come, later)
# Tableau V8
# It ties like Rubenesque Woman at MOD13

Christian Chabot

Elisa Fink

On Being a Data Scientist-- the Warren Jeffs Axiom

2013-05-16T16:10:00.001-07:00

May 16th, 2013 (MVCA)
I lost my remaining respect for the Dignity of the American Populace when I read that Gwyneth is the most beautiful woman in America. Hmmm. I might need to start reading US again.

Two Apples over the moon in love with Shakes...
# Data = Boob-Raider
# Tableau V8
# it ties

Marianne Pearl

Lara Croft

Jane Smith

Lola

Gina Malacici

Lisa Rowe

On Being a Data Scientist-- the Amazing Rack proof

2013-05-16T15:01:00.002-07:00

May 16th, 2013 (MVCA)
I've had Angelina Jolie on my mind ever since I saw her in that movie, "the Model who goes on a bender"-- as if there are any other types.

In my most manly opinion, they should have given Mrs. Brad Pitt an Oscar for portraying how breasts should be rubbed against a chain-link fence-- they gave an Oscar to the other Mrs. Brad Pitt for playing a severed head in 7, so mock me all you want.

Blessings and Gratitude for Mrs. Brad Pitt's dearly departed-- they were amazing
# Data = Boob-Raider or (CarlSagan^50)
# Tool = Tableau v8
# Math = X^3 or something like that

Fake Polynomials

X and Y^1

X and Y^2

X and Y^5

X and Y^7

X and Y^580

X and Y^581

(Ain't no bees without honey, ain't no milk without a cow)^590

On Being a Data Scientist-- the Mating Dragonfly Conjecture

2013-05-13T18:58:00.001-07:00

May 13th, 2013 (MVCA)

I saw a butterfly dying smoothly on a stick and she seemed so content in the calm wind of life that I let her be and wandered down toward the creek to take a sip and pander the mathematics of change.

As Infinity approaches Zero she understands all things precious and small bits delightful
# CarlSagan^50
# else

Butterfly

Dragonfly

On Being a Data Scientist-- the Michelle Pfeiffer Proof

2013-05-13T12:19:00.004-07:00

May 13th, 2013 (MVCA)

I had always wanted to build the Ramp of Perpetually Infinite Motion and being of working-class decent I understood that some dreams are just made to be dreams-- I certainly do not possess the traits of Musky Elan when it comes to the desires of Prometheus and all the other Titans who dared to will beyond the Gods.

As it is, I am just a human guessing wildly about notions of this and theories on that, and each morning after I've had my stroll and cup of coffee, I sit down and poke through the rubbish from the night before.

I asked Cat Woman out on a date and she was startled by my intentions, so she filed a complaint
# Data = CarlSagan^50
# The Cat's Meow = Y^2 + a1XY + a3Y
# It ties like kitten's whiskers
# Tool = Tableau V8

I'd 'hit' a Super Hero

Latin Woman are full of bumpers

I fear those of lesser powers

On Being a Data Scientist-- The Steven Segal Axiom

2013-05-12T20:16:00.000-07:00

May 12th, 2013 (MVCA)

Did you tell somebody that you love them today?

I did, and it felt ...

Data
# Rows = CarlSagan^50
# Math = Y^2 + a1XY +a3Y
# Tool = Tableau V8
# It ties

Love

Prosperity

Denial

On Being a Data Scientist-- the Mother's Day Theorem

2013-05-11T10:52:00.000-07:00

May 11th, 2013 (MVCA)
I only see my Mother a few times a year, but my life is filled with all that she has ever [fill in the blank] me.

I see my Children's Mother as often as I can, and I am blessed to have met her and I think she is pretty nifty.

So, in appreciation for all the other Moms out there, I've decided to explore a few of the words that came to mind when I created the Mathematics of Mom.

Mother's are... (Yes, I made them Possessive of Being)
# Rows still equal CarlSagan^50
# Mom = Y^2 + a1XY + a3Y
# it ties the family together

She always smiles when she sees me

I cry when she is away

I think of her when I'm in pain

I am speechless when she is near

She taught me how to love all that I might fear

On Being a Data Scientist-- the Sperm-Whale Steak Axiom

2013-05-10T10:08:00.000-07:00

May 10th, 2013 (MVCA)
I went to the Natural History Museum of Art and looked into the eye-sockets of a whale.

I could see the joy of swimming in deep, open water bleached white to her sandy bones, and what remained to construct her form was my solitary wish to snuggle up against her and feel for her once pounding heart.

Never fear to love another.

She is liquid beauty packaged for consumption
# Rows = CarlSagan^50 by Mobile 'Dick' Melville
# = 3X4 - 2
# It ties like a scrimshaw ribboned to love's worthy hunter

On Being a Data Scientist-- the Dying Grandma Axiom

2013-05-09T19:55:00.000-07:00

May 9th, 2013 (MVCA)
I happened upon a friend who was off to Australia to celebrare her Grandmother's present passing into death, so I granted her my obligatory and unintentional-insult.

"Oh shit, it a'int like she's Sixty."

Which reminded me of my other friend who was using his dead Mother's money to take her Grandchildren to Austria for a summer visit.

"My goodness! When did she die?"

Certainly, the value of friendship is forgiving of such transgressions and as such, I offer these reflections on the Finite instance of life as I can't help but note that that was too many A's for my liking.

She is always Alive within me
# Rows = CarlSagan^50 and as Finite as Grandmother's in Australia
# Math = Y^2 + a1XY + a3Y
# It ties like Salt Water candy

My Grandmother loved Entenmann's and Puzzles, but Taffy always bothered her Bridge

Step 1, Take a bite

Step 2, Chew politely

Step 3, Only play Penny Slots

Step 4, and Always use a Pen

Step 5, Solve the easy ones first

Step 6, Only trust Black Coffee lovers

Step 7, Never marry before Sex

Step 8, Always make-up before bed

Step 9, Secretly sip your sins

Step 10, Don't retire with people your age

Step 11, Champaign-Urbana

Yum

On Being a Data Scientist-- Benoit Mandelbrot and the Jump Jive

2013-05-02T13:50:00.000-07:00

May 2, 2013 (MVCA)

When I was in Second Grade, Pops came to speak at the "Bring your father to school day."

I remember being surprised when he suddenly showed up at school, but the moment he began walking toward the front of the class, I felt happy.

Back then he always wore suits and cowboy boots and so he made quite the entrance. And then he began speaking to us all.

"I'm x10ba's father, and I build ..."

By the time he left, he had pulled out a full arsenal of kid-friendly delights (ranging from balloons and string all the way through to the chocolate-chip cookies that were made that morning and into propulsion, vectors, the Moon, space and an idea regarding a crumpled piece of paper.)

The sound of Cowboy boots on tile floor
# Data same Mandelbrot dataset
# Rows = CarlSagan^50
# M=X^2+C
# It ties like Bough-knots on Xmas Morning

Ezra Pound peered at the sky through cracks in his cell

1.x10ba

2.x10ba

3.x10ba

4.x10ba

5.x10ba

6.x10ba

7.x10ba

8.x10ba

On Being a Data Scientist-- Benoit Mandelbrot had the Big 'H'

2013-05-01T13:27:00.002-07:00

May 1, 2013 (MVCA)

Happy May-Day everybody!!

Gnoc' and Tortelin'
Same good ol' Mandelbrot
M = X^2 + (i)

Ezra Pound, "she ties like a kimono"

Sicilian-Pisan Pie

P(Tan) Giro

Bu Z*N Tour

On Being a Data Scientist-- Benoit Mandelbrot and the NYC

2013-04-29T20:01:00.000-07:00

April 29, 2013 (MVCA)

From time to time, the Gods of yore were known to be quite the bastards or bitches, and when these

Gods came into contact with a mortal who was as infamously as bastard-ly or bitch-elly, well, great stories were born.

Transgression regarding the Empire State

For instance, Clark Kent, the great American anti-hero, had such a horrible drinking problem that when he'd encounter another 'mean' drunk in Hell's Kitchen, the two would, after hours of jousting and out-drinking, end up on top of the Empire State Building trying eagerly to out do each other with some elaborate ruse or another.

Ultimately, As the two mean ol' drunks looked out upon the dawning city night, Clark Kent would dare,

"The gusts of Midtown are soooo great (hic), that you can leeeeeap off from the deck (hic) and the elegant Nor-easter (hic) will carry you right back up to where we are (hic-cup)."

Clark Kent would then, of course, demonstrate the feat (hic), leaping out, floating up, and landing nicely back on the deck ready for another round of Crack margaritas or whatever else may be available (hicc-up).

Of course, the mortal (hic), being equally bastard-ly (hic), would try to out-duel the other bastard (hic) and would climb up (hic), leap out (hic) and ...(hic-splaaat.)

Revealing the punch-line,

"You are a F'n mean ass drunk, Superman."

Which leads to notions regarding my equally bitch-elly bender (damn you arrogant PHds!!!!)

A beautiful bitch tried to out-spin a bitch of a spinster, and in reverence, an equally mean ol' bastard penned the notion (hic),

"Arachne, che mi porta Fortuna;"

The butterfly has gone out thru our smoke hole [sic]

Same good ol' Mandelbrot non-such as before, but now with even more iterations!! (Yeah for thee)

M still = X^2 + C

N now = 161,606 X 6

Tool still = Tableau V8

Dank now = Questionable Ratios abound

And yes, it ties

Inksters are delicious

Hornets are Horny, Spiders are Salacious

I love the Collarbones of buxom women

On Being a Data Scientist-- Benoit Mandelbrot's Canto

2013-04-28T16:15:00.003-07:00

April 28, 2013 (MVCA)

I was deep in my boredom of thinking about 'Uncle Bob' and how he was probably the 'bastard' child of Ezra Pound when I figured I should just leave my prose-minded-mind behind and turn to the numbers.

Data
M = X^2 + C
N = 161,604
Tool = Tableau V8
Dank = It Ties

Mandelbrot enjoyed him some Drank

On being a Data Scientist-- PCA of PiPhi

2013-04-11T20:40:00.000-07:00

April 11, 2013 (MVCA)

Back when Cigarillo or G.Dumya's Pa were in the House, I spent my Senior year of college at Cal. I wasn't registered there, but I found the classes to be interesting so when I wasn't hanging out with my G-friend, I was pretending to be a Bear and slipping in and out of grace.

We lived down on Ward Street in a flat that had Bauhaus fans on top and Cure fans below and on the weekends we'd go to the various Frat parties and such-- my G-friend was a Pi Beta Phi.

Of course, I failed all my 'regular' classes that I was supposed to be in at my 'regular' school, but I found my time at Cal to be somewhat rewarding and stimulating and didn't mind so much that my GPA went from ~3.9 to 3.5 because of 3 classes, and then of course there were the 'life lessons.'

The Golden Ratio
One day, just after my G-friend attended her morning classes, I was left to my own devices with my G-friend's roommate.
About 30 minutes into a discussion regarding Kaos theory, she got up and retired to the other room-- this wasn't surprising to me since it happens quite often to me when I discourse with people-- and so I went back to doing whatever it was I felt I had to do.

"X10ba?" my G-friend's roommate called out to me, "Can you come reach something for me, I'm a bit too short."

When I went to assist the Damsel, I soon found myself in a rather delicious predicament; you see, my G-friend's Roommate had decided that it was the appropriate time to pull a 'Mrs. Robinson' on me.

Pi - Phi^2
When she released her towel and revealed all that I would only ever imagine, we immediately engaged in a discussion regarding Lust and Fidelity. Needless to say, I only have a few regrets, and my G-friend's Roommate is number 2 on the list...

PCA of PiPhi
Principal Component Analysis of Pi and Phi or, "There's Gold in that Valley."

Data
Same as before, but now with Phi

Logic
I love beautiful women

Findings
Orinda, CA is the 455 Capital of the World

PiPhi(x10ba)

Ward Street House(x10ba)

Kaos Fury(x10ba)

Mrs. Robinson's Hot Tower (x10ba)

On being a Data Scientist-- PCA of PiE

2013-04-10T20:39:00.000-07:00

April 10, 2013 (MVCA)

I partially relate to James Stirling in that he was expelled from something for being 'different' and he liked to get his 'drank' on down at the Brig o' Turk. From there, I figure anything else would just be HBS...

Stirling's Close Guess

From what I can gather via the mathematics, James Stirling had a great appreciation for Pi (I am assuming all types) and he tried to figure a way to put the 'e' back into it-- Numbers, Permutations, Hotties, Ho's and Approximation, oh my.

PCA of PiE

Principal Component Analysis of Pi and E (read 'e' the mathematical constant) or "Ain't PiE Grand!"
All work completed in Tableau V8 and needs to be vetted against SAS, R, Matlab, etc.

Data

# 1000 rows of Pi and 1000 rows of e

# X (x1,x2, ... x1000); where x1 = 3, x2 = 1, x3 = 4, etc.

# Y (y1,y2, ... y1000); where y1 = 2, y2 = 7, y3 = 1, etc.

# Sqrt(N) FuckData!

# Covar-Matrix and Eigies are listed on the Viz

Logic

# I needed some data to perfect my process, so I found e and 'mixed' it to Pi

# X = sequential Pi (int) starting at the 3 and continuing to count 1000

# Y = sequential e (int) starting at the 2 and continuing to count 1000

# I rounded to (decimal, 10), but it ties (for the better part)

# I created some 'Questionable Ratios' for the fun of it

Findings

Pi is 'extra' delicious when e is added

PiE(x10ba)

Stirling's Permutation(x10ba)

Spouge's Proclamation, "Puss-n-Boots = a peculiar man"(x10ba)

CreamPiE delight(x10ba)

On being a Data Scientist-- PCA of Pi

2013-04-09T15:04:00.000-07:00

March 14, 2014 (MVCA)

In honor of the peculiar man who told me, "You can't do anything in Tableau!" I offer up the next international Pi day flag of salute.

PCA
Principal Component Analysis of Pi as rendered in Tableau V8

Data
# 1000 rows of Pi (though I can expand to 1M)
# X (x1, x2, ... x1000); where x1 = 1, x2 = 4, x3 = 1, etc.
# Y (y1, y2, ... y1000); where y1 = 3, y2 = 1, y3 = 4, etc.
# N FuckData!

Logic
# I needed some data to test, so I found a dataset of Pi
# X = sequential Pi(int) starting after the decimal and continuing to count 1000
# Y = sequential Pi1(int) starting before the decimal and continuing to count 1000
# I rounded a lot, but it ties (for the most part)

Findings
I like PiE (especially boysenberry, peach, lemon and cherry) and Pi likes me; now if somebody would just vet my results in something like SAS, R, Matlab or any other Statistical Software I could settle the aforementioned dispute.

Fig Pi(x10ba)

On being a Data Scientist-- the AMZ index

2013-04-07T09:10:00.001-07:00

April 7, 2013 (MVCA)

Back when I used to oversee the 'finances' at walmart.com, as part of my duties I would present a weekly forecast for Sales and Margin as well as a monthly budget down through Operating Profit and into a P&L.

At the time, this was 2004, the Dotcom were 'terrified' of CostCo and Amazon and we were still recovering from the Jean Jackson mistake, so the only support for eCommerce came from the myriad of McKinsey decks, MBA white papers, whatever else we could cull from the numerous resources (Forrester, Yahoo finance, etc.) and a $40M yearly cash injection from the MotherShip.

As a collective we digested this information and did our best to spend as little as possible, while attempting to hit our goals for the year.

One day I read a McKinsey deck that said, "Walmart.com is a $100B opportunity with the correct CRM strategy."

There was some more non-sense attached to the 60 page deck about how McKinsey would drive the results and there was a $1.6M bill for consulting services, so I ripped off the 1st few pages and then threw the rest of the deck in the trash and asked myself,

"Is it possible for Dotcom to be 40% of all Walmart US sales?"

Being the crafty sum-bitch that I am, I figured I better get ahead of that game and so I devised a plan and created a simple model.

The AMZ Index
It became clear early on that the information that we were digesting wasn't good enough-- MBAs generally don't know what they are doing, Consultants just dress nice and talk a big game, Quarterly releases are just masturbatory handshakes between the SEC and the people, and the Planners and Merchants at the Dotcom were mainly 'fearful' of finance [the first CFO was Pen15 and the last one was hooked on Prescription Narcotics, Dorritos and Pepsi and would fall asleep in close meetings], so I understood the business' reluctance to be honest and forthright with knowledge.

I had the great fortune of sitting with the Merchants and as a result they soon accepted me as being one of them; however, this really did nothing for my models as any time we would discourse we would just bullshit about sports, stocks and who was banging who and where.

I also had the fortune of sitting with the Executive Leadership and as a result I was soon pelted with numerous questions and requests; however, this really wasn't that rewarding since my hatred for PowerPoint was akin to my hatred for the Cs.

As I sat amongst the tribe (leadership to the right, workers to the left) I noticed a pattern in behavior.

The Stock Clerk (the guy who brings packages and such to each person) began appearing on our floor (5th Floor Marina Blvd, baby!) at around/about the same time every day. And being the Data Scientist that I am, I began cataloguing (first on paper then into excel) the daily package delivery to Senior Leadership and the Dotcom Merchants (home and electronics).

At first (2004), I was curious in terms of distribution during the week-- i.e. which day is heavier and which is lighter-- with the hope of understanding how packages are delivered and whether or not I could leverage something within the Walmart Supply chain to increase the Dotcom profit.

Noting that I understand that the sampling is small, but as a Data Scientist I understood that the pattern existed and thus might be meaningful insight for our own business.

CostCo is an '3vil-do3r'
Several months into my data collection, I expanded my analysis to include distribution of vendor (Costco, Amazon, Walmart.com etc.) and that is when I discovered something that was odd and I asked one of the executives a question,

"Why do we use Costco as a vendor for our internal office supplies?"

Needless to say, a 'Marketing' memo went out and required that we no longer use Costco to purchase items as seeing an idling Costco Truck in front of a Walmart Corporate Office was bad PR.

Feeling dissatisfied by the executive orders, I continued my collecting and created a pro-forma for the CostCo exclusion-- they should just spin-off Sam's Club already.

The Gold Standard
After about 7 years of collecting, I retired the collection of the data and inserted the following assumption as 'fact' into the algo. Noting that seasonality skewed counts of packages, but distribution remained relatively stable.

Walmart.com Merchant and Executive Leadership Buying habits as experienced through packages delivered to Wamart.com (sample data collected from 2004 - 2011, Brisbane, CA and San Bruno, CA).

Distribution by Company of origin of product shipped to Walmart.com office
Amazon.com: ~90%
Clothing (Piperlime, Eddie Bauer, Lands End, etc.): 5%
Walmart.com: 3%
Misc (Costco, etc.)/rounding: 2%

Applying the logic
I always found it odd that the Walmart.com Buyers and Senior Leadership shopped Amazon, but the data was irrefutable and supported the McKinsey Deck-- issue being that the consultant just copied and pasted the Walmart.com logo over the Amazon.com logo and charged the Walmart US $1.6M.
Nicely done indeed.

BTW, I placed my first Amazon order in the Fall of 2012, which was several months after I said 'good' buy to the Dotcom for the last time.

A question to the Amazon.com Data Scientists.
Did the implementation of the Amazon Tax in California impact shipments of goods sent to customers at 7000 Marina Blvd, Brisbane or 850 Cherry, San Bruno?

On being a Data Scientist-- 5 Steps to becoming all that you are

2013-04-01T13:16:00.003-07:00

April 1, 2013 (MVCA)

The following steps are provided free to you and do not require an advanced degree.

1. Get Data (connect, find, mash, hack, whatever...)
2. Vet Data (tie, foot, align, match, whatever...)
3. Carve Data (analyze, regress, pivot, PCA, visualize, whatever...)
4. Understand Data (be who you are)
5. Fuck Data (Algo, Machine-learning, monetize, whatever...)

If you are too stupid to follow these simple processes then I might suggest that you take a career in Executive Management or pursue an MBA from the IVY.

Iaintjudgin(x10ba)

On being a Data Scientist-- How to analyze a Petabyte of data on a laptop (The Cherry Filling Theorem)

2013-02-21T20:12:00.000-08:00

February 20, 2013 (MVCA)
All I ever knew of good ol' Benford was that he was a dude who played around with mathematics and such, but in a lot of ways I figure he was pretty much like you or me: he probably tried to get laid, he sought housing of some sort, and he tried to figure out how to get food on a pretty consistent basis. From there, I figure it would just be conjecture, and I'll leave that nonsense up to the Elies and the Crimson Snobs.

22.9 Gigabytes of pitted cherries
The Data Scientists that I know like to talk about rows, matrices, vectors, databases, clusters and all sorts of other nifty ideas and toys. I like to be around other dudes who don't need to WIP out their decks or flex their muscles over the tiniest of misunderstandings and so the men I know (like Pops) tend to be erratic with their thoughts and tender in their emotions, ideas and passions. So, forgive me as I give a little back to my fellow wo/man as I allow bits of notion to leak out as I search for a bit more clarity within the mathematics that are far too expansive for my meek mind.

Details
Datasource: Sequenced data pulled from Teradata (select * from table1 where (filtering clause))
I used the filter to restrict the size of data that was coming out my way
Rows: 413,928,565 (it ties back to original table1)
Size: 22.9 Gigabytes (24,642,014,573 bytes)
Time to export: 280 minutes
Column/Character width: 59 (excluding delimiters and nulls)

How to get laid during Benford's time
I had to look up some history on Benford, and what I understood is that he lived from the late 1880's to about 1940 something-- several world wars and other non-such happend during his lifetime as well as having limited ability to travel and communicate with other ideas except via discourse or book. And, I imagine that as this dude went out and about, he tried to fancy a lady or three with his parlor-tricks in math and other advanced knowledge. And, I am pretty sure that he had limited access to a computer and thus had to solve all of his math in his mind or on pad and paper. How lucky I feel.

When I'm not at work, my math isn't very good
So forgive me when I try to estimate some peculiar math as I attempt to analyze a Petabyte locally upon my machine.

I read that a Petabyte is equal to 1,000,000 Gigabytes. Given that I have 23 Gigs (rounding) imported into a Tableau Data Extract on my MAC, I figure I need another 44,000 (rounded) TDEs and MACs. Hmmm. I worked for walmart way too long to have any amount of cash sitting in my bank, so I figure I can't make that happen-- not withstanding the fact that it would be a terrible waste of resources. [I bet there is a data-center or three filled with young university students ready to tackle any sort of work or problem-- talk about a distributed system.]

I also figure it would take about 12,000,000 minutes to port all 44,000 instances onto local machines. I'm not sure I have that sort of time, but given that code is timeless, I figure somebody much smarter than me will devise and implement a process that would minimize that hindrance.

The Cherry Pie Theorem (as loosely worded as my women)
I wasn't satisfied with Benford's Delight as the only thing it told me was that Truly Big-data is flat and emotionless, and that notion didn't seem fair to some of our beloved heroes.

I analyzed 6 columns/characters by 413M rows of data. The 7th column pooped out and couldn't take anymore. All work completed on TableauV8 via MAC VMware.

Instance 1(x10ba)

Instance 2(x10ba)

Instance3(x10ba)

Instance4(x10ba)

Instance5(x10ba)

Instance6(x10ba)

On being a Data Scientist-- How to analyze a Petabyte of data on a laptop (The Samson Axiom)

2013-02-19T19:35:00.001-08:00

February 18, 2013 (MVCA)
Several years ago, I had the fortune of going on a 'bender' with my best-friend. The whole mess started once we stepped over the actresses, groupies and strippers that had spread themselves neatly on the sidewalk in front of the Viper room; a wailing bunch of hopefuls gyrating and grinding feverishly into the ghost of hunks passed.
Our delirium ended just after we tried to step over the sleeping body of 'Stuart Carson's' father, our motions too drunken to quiet the man, and he popped up suddenly and began singing,
"I hear the train a comin...'"

Having escaped the sonic-massacre, my best-friend and I had a hunger so fierce that... well, you get the idea, and we went to the kitchen and looked in the pantry, fridge and cabinets.

Being Bolognes', my friend is incredibly endowed with the ability to cook, and that night he took a pound of american pasta, some frozen vegetables, a half-limp zucchini, quarts of v8 juice, some salt and a ton of butter and cooked the whole thing off a hot plate.

"It doesn't matter the ingredients, it matters how you cook them." He announced as he downed our last beer.

Three servings later, I was still licking the plate clean and begging for more.

But, those were different times and now he has moved on to cooking with beautiful food, better utensils and better entertainment and I still lick the plate clean and beg for even more.

The Haute-plate
I can't get a peta-byte locally to my machine. Those are physical, irrefutable facts.
I can hit a peta-byte of data on Teradata, and I can return that data into a nice, delicious box of goodness, but that isn't fun.
I've been on this bender for so long that I can't recall when I started, and at the moment I feel a sense of pride in trying to Data-scientry with what I have: a shitty-ass MAC, 22.9GBs of data and Tableau.

Sonic-master piece of pie
Somebody I don't know asked me to plot Benford's law on top of the data. 3 Hours later, I came across the following answer. [note, Benford only accounted for 1-9, but computers (darn them) account for 0-9, so I adjusted.] It ties. I removed that annoying 'null' as well (next time I'll use a coalesce to correct.)

Say what you will, but Tableau is dank with the big-data.

Benford's delight

Note
# Green is key lime pie, Red is Benford's delight (adjusted)
## the shapes are 4 dimensions of the sequence and Benford's cherry pie

On being a Data Scientist-- How to analyze a Petabyte of data on a laptop (The Crust).

2013-02-17T21:42:00.001-08:00

February 17, 2013 (MVCA)
As a child, I had the great fortune of being a test-pilot for all of Pops' culinary explorations-- as long as the recipes contained flour, sugar, milk, eggs and whatever else Pops might have found to be delightful and fresh, he would stand in the kitchen for hours, kneading, rolling, cuisinarting, whisking, folding and baking, all the while I would wait and wait for the timer to BING as the waft of amazing goodness filled our house.

I've eaten Peta-cuts of Lemon Meringue, Strawberry Rhubarb, Apple, Cherry, Pumpkin, Pecan and Sweet Potato pies, I've eaten Giga-bites of cookies, morsels of sugar, brittles, candies, cakes and I've licked bowls and spatulas and I've had peach ice cream that was so sweet that I could still taste the flutter of little lustful bees. Yum.

I rarely saw Pops' measure his ingredients, but I understood that he baked with the same self-confidence that helped put Neil Armstrong and friends on the moon. When we would eat the delights, the house was always silent except for the constant chatter of the television, the clink and scrape of fork and spoon on plate, and the guttural moans and groans of absolute, exquisite delight.

The one and only time that Pops' made a mistake I told him, "The pie was a bit too runny and not tart enough for my taste."
"Don't fuckin' eat it then" he said as he slammed his plate down and stormed off to be alone. I still cry from that memory, and I am still embarrassed that I said that to him, but he forgave me and, out of respect, I never did eat another cut of that particular pie even though it wasn't as bad as I had described.

The ingredients
System: MAC Laptop, 2.3 GHZ, Intel Core i7; 8GB, 1600 MHZ DDR3, OSX 10.8.2; 250GB storage, VMWare Fusion running Tableau Version8 (which is in beta)
Data: 22.9 GBs (24,642,014,573 bytes), 413,928,565 rows and 59 Characters per row-- not including nulls or delimiters

I decided to throw a few notions against the test dataset to see what would happen:
#1 avg(len(str(column1)))
#2 countd(column1)
#3 right(str(column1),1)

The Crust
As Pops will most certainly attest, a good crust always makes the pie regardless of filling.

I ran formula #1 against the dataset and I added a 2 attributed dimension onto the shelf. 46 minutes later, Tableau returned the result that I expected.

I ran formula #2 against the dataset with the same 2 attributed dimension and 6 minutes in, I received the following error:
Tableau Data Engine Error: 4 :Hashgrouping::InsertHash:Collisions = 423268, Size = 42326783, rate = 0.01

I figured that the dimension might be adding a complexity given the size of the data extract, so I removed it. 6 minutes in, I received this error:
Tableau Data Engine Error: 4 :Hashgrouping::InsertHash:Collisions = 846429, Size = 84682843, rate = 0.01

Seems I was 'right' about my measurement, in a way, and I understood that the Countd was too taxing-- it is even taxing in Teradata, so I wasn't too disturbed.

I ran formula #3 against the dataset, put the result in the oven and 46 minutes later, BING!!!!

Absolutely delicious. Tableau is certainly dank and totally impressive on the Big-data

On being a Data Scientist-- How to analyze a Petabyte of data on a laptop (Key Lime Recipe).

2013-02-16T21:45:00.001-08:00

February 16, 2013 (MVCA)
Since @tableaudatanerd is reluctant to part with either of his Lime Green Shirts, I've decided to offer a new wager: (Two things that go wonderfully together) A byte of Key Lime pie (Gluten Free of course) to me if I win, and if I lose then a beautiful glass of bourbon (assuming that alcohol is shippable across state lines) to be shared with a fellow nerd at the next TCC.

The 'Ground Rules'
I acknowledge that CMU is utilizing a Mac mini to out perform 1636 nodes on Hadoop and thus, in essence does what I am about to do, but I'll let them work with the 'smart' stuff and I'll stick to the good old bread and butter inherent within the working man-- there is no P H or D in x10ba.

And, given that I barely graduated high school, did not study Computer Science in college and got expelled from art school, I acknowledge my weaknesses and will not attempt to build a fancy algo or machine process to help me in my quest-- mainly because my coding techniques would probably spread a world wide virus of which Anonymous would be impressed, and I can't really afford any additional heat from the FBI.

So, I'll just stick to the basics:

1. find a Petabyte of data
2. boot up my laptop
3. cram said data into Tableau (this will be hard since I have 1TB of space only)
4. analyze said data
5. enjoy a slice of pie
6. move on to the next project

Extra 'Credit'
Because so much is currently at stake; I really love pie and I hate being publicly ridiculed, I will see if I can pull this off without too much of my sardonic 'wit' polluting a good ole friendly wager-- achieving this would result in a nice dollop of cream for my pie, and failure would require me to drink my bourbon with 3 cubes of ice.

The pie will 'tie'
Additionally, given I found it hard to sit through college courses that involved 'theory' and such, I will use data that is actually 'real' and 'viable', thus allowing my final result to be tested against the source. Noting that coming by a Petabyte of data isn't that difficult of a task, but that sharing that data publicly is probably not advisable, so I will add in the extra step of establishing a control and anonymizing the data that will be blogged, posted and discussed. Whew.

Now I just need to figure out how many rows are in a Petabyte and I can be on my way :)

A peta-bite of Key Lime Pie (x10ba)

On being a Data Scientist-- How to analyze a Petabyte of data on a laptop.

2013-02-16T05:55:00.001-08:00

St. Valentine's Day, 2013 (MVCA)
I like 'challenges' more than I like pie, and, as such, I've decided to embark on a new little Pie-diddy that I will slice up and devour-- noting that I have an even 'better' project that I've almost completed and am about to upload to the joy and wonder of all my fellow data scientists over at the Twitter, the Facebooks, Pinstagram and +FlowingData

The 'challenge'
Can I analyze a Petabyte of data on a laptop?

‏@TableauDataNerd

@x10ba The @tableau Data Engine isn't a distributed compute engine like @Teradata or @hadoop. You can't analyze a PB of data on a laptop.

Details

The 'wager'

If I lose this wager then I'll offer up my humility to the scorn and ridicule of all the interwebz can throw my way. However, If I should win...

‏@x10ba

@TableauDataNerd @tableau @teradata @hadoop I'll betcha your lime green shirt I could analyze a PB on my laptop :) Give me a @PETA-moment.

Details

The 'acknowledgement'

Robert Morton is much, much smarter than me and his understanding of the 'limitations' of Tableau and the Tableau .tde obviously surpass my meek and humble ways. I am a huge fan of @tableaudatanerd and an avid supporter and user of @Tableau, so any notions that I might make over the next few posts are done so with admiration and respect.

Full disclosure

I did have a recent moment of 'intellectual-stalking' @tableaudatanerd, but I did so without the understanding that we would come this far...

‏@x10ba

#tcc12 #shauncassidy talkin about queries I bet his hair smells amazing and he has nice posture. Not my 'type', but I like quickies.

Expand

‏@x10ba

#tcc12 Waiting to connect. Dude is more like #stevetyler w/o the dress. He has a nice voice and I look forward to some cuddling, perhaps

Expand

‏@x10ba

#tcc12 #stevetyler is a'ight. Knows his db. Grammy/Emmy worthy if not for @katyperry-- she is my type. #tableau

Expand

One lime green shirt coming my way (I wonder if the sultry brunette comes attached as well, boy I hope so :) )x10ba

the secret code to my thoughts

2012-12-11T18:00:00.000-08:00

The answer is right here.

Big Data Sciences-- Dandy-lion to her whispered might

2012-12-06T19:07:00.001-08:00

December 6, 2012 (MVCA)
I've received a few bits of 'fan mail' regarding my visualization of a Billion [row]s, and in the interest of remaining irreverent, I'll share a few of the more 'interesting' ones from a few of my 'followers' and offer a response to those ever-enquiring minds.

Fan Mail
"I doubt that you can even count to a billion, so stop filling the interwebs with this drivel! By the way, I know billions you dumb fuck. Now get back to work." Zuckey (Zuckey@thefakebooks.com)

And from another follower down in Shallow Alto,

"Are those glazed donuts, or some of them fine Red Velvets?" B##### (Sheryl@facialbook.com)

And another couple from #Buffet's elite,

"We keep our prices low, our dividends in the family and our wages criminally below minimum." Greg and Marisa (cuckold@bigprofit.com)

And a final one from a well-known thief,

"Can you send me the source file? I want to get promoted." Padley Painblo (ppainblo@wall-wart.com)

My 'canned' response
Datasource: 1Billions [row]s by 18 columns.

3(x10ba).1

3(x10ba).2

3(x10ba).3

3(x10ba).4

3(x10ba).5

3(x10ba).6

3(x10ba).7

3(x10ba).8

3(x10ba).x10ba