Musings, Rants and Ponderings Of A DB Architect

RAM prices are totally out of control, tripled in less than 5 months

2025-12-05T10:38:00.000-08:00

If you have been paying attention, you have noticed RAM prices are out of control

Take a look at this order I placed on July 20th, so 4.5 months ago

I paid $168.99 for 64GB of RAM. Four weeks ago, it was $297.99 so almost double

What is it today? Check it out, here is a link to Amazon so you can see the current price https://amzn.to/43irnfi

Today, this is what is listed as the price

That's $537!!! That's more than triple, is that not insane??

So what happened?

On October 1st OpenAI signed two simultaneous deals with Samsung and SK Hynix for 40% of the worlds DRAM supply... the shock wasn’t that OpenAI made a big deal, no, it was that they made two massive deals this big, at the same time, with Samsung and SK Hynix simultaneously! In fact, according to our sources - both companies had no idea how big each other's deal was, nor how close to simultaneous they were. And this secrecy mattered. It mattered a lot.

Had Samsung known SK Hynix was about to commit a similar chunk of supply — or vice-versa — the pricing and terms would have likely been different

Read the whole thing here https://www.mooreslawisdead.com/post/sam-altman-s-dirty-dram-deal

The good thing for me was that I sold the 32 GB stick I swapped out for the 62 GB for about the same price that I paid for the 32 GB :-)

LinkedIn is more and more green these days and not in a good way....

2025-05-26T06:14:00.000-07:00

If you browse LinkedIn these days, you can't escape the fact that a lot of profiles have a green #opentowork badge. I know a bunch of these people personally and know that they are technically very strong. But it's looking bleak at the moment, uncertainty with the tariffs and the overall economy has forced a lot of companies to not even give earnings guidance. These same companies have also instituted a hiring freeze for the foreseeable future.

Then there is the elephant in the room... AI. Perhaps 1 person + AI can now do the same work as 2 or 3 people before? Time will tell but for sure it's a factor to take into account in regards to all these layoffs

This period right now reminds me of the period around October and November 2001. First you had the dot com crash in 2000, this was followed by the September 11 attacks which caused a massive stock market crash, all this resulted in massive layoffs.

Back to current times: If you check the layoffs.fyi site, you can see a lot of layoffs in Tech and Federal Government

Here is an example

Let's go back to to the year 2001 again: I used to work in Silicon Alley, this was in New York City around 23rd Street and Broadway/5th Avenue, you might recognize some of these companies that had offices there like RazorFish and Kozmo.com.

I was a web programmer doing ASP/JSP/ColdFusion and SQL Server. Life was good in the summer of 2001, so good in fact that we used to have this massive 3 year project plan printed out and taped to the wall. Then in September someone took it off. A few weeks later we were called into a meeting and were told our last day was today. So of course the first thing we did was buy some beer and huddle together to says goodbyes and reminisce about the past few years.

Now it was November, who will hire anyone in November? I decided to have 2 types of resumes, one was a Web Developer focused resume, the other was a SQL Server focused resume. I uploaded my resume to monster.com and a few days later someone contacted me that they had a 2 month contract job that needed someone with SQL Server 2000 and DTS skills. I decided to take it since like I said earlier, nobody will hire you in November/December. Well I got the job, after 2 months, they extended it for 3 more months, after that, they made me a full time employee. I stayed there for 3.5 years.

Why am I telling you this? Most people will not take this 2 month contract, but you never know, it might get extended like in my case, or you might learn a new skill during that time that might help you land the next gig. In addition, the co-workers at this temporary job might know someone that needs a full time job filled

A few other things that come to mind:

Customize your resume and cover letter for each job. Highlight the skills and experience that match the job description exactly. Use keywords from the job posting: many companies use automated systems that screen for them.

Ensure your LinkedIn profile is up to date and reflects your current skills, accomplishments, and job goals. Recruiters often search LinkedIn before reaching out. Use a professional photo, write a compelling summary, and showcase measurable achievements.

Reach out to people in your industry: friends, alumni, or professionals on LinkedIn. Ask for informational interviews or referrals. A warm introduction often leads to more interviews than cold applications. Ask for feedback about your resume to these people as well as endorsements on LinkedIn

Jobs often receive the most attention in the first few days after posting. Apply early, and focus on roles you’re genuinely qualified for. Quality over quantity improves your chances of standing out.

After applying, follow up with a short, professional message to the recruiter or hiring manager if you can find their contact. Reaffirm your interest and briefly restate why you’re a strong fit.

If you are 55 or over, do you know about the Rule Of 55?

Ageism is a thing, companies might say you are overqualified or use some other excuse not to hire someone who is older. For those that are 55 or older, keep reading.

The Rule of 55 is an IRS provision that allows individuals to withdraw funds from their 401(k) or 403(b) without the usual 10% early withdrawal penalty if they leave their job in or after the year they turn 55. This rule applies only to the retirement account from the employer they just left, not IRAs or accounts from previous jobs. It’s designed to offer more flexibility for those who retire or are laid off early. While the penalty is waived, regular income taxes still apply to the withdrawals.

See more here at the IRS site: Retirement topics - Exceptions to tax on early distributions

This can help you out if you are in a cash crunch since there is no 10% withdrawal penalty, you do however still pay regular income tax on that amount

Some ideas for 2021, books to read or to get, stuff to plant, exercise to do

2021-01-09T12:45:00.001-08:00

Here are just some ideas of what I would like to do in 2021, these are all non professional goals if you will

Run 800 miles

I ran 800.3 miles last year, this should be easily doable, it's about 4 miles 4 times a week

Read 50 books

Last year I read 60 books.. I assume at some point, I will be back in the office two times a day or so, this will maybe bump it down to 50 books from 60 books

Books in the queue

These are books I or my kids already own, some of these I have gifted to them this Christmas season (Masters of Doom, 1984, Brave New World, Infinite Jest, Pirate Latitudes)

I have not read any of these except for 1984 and Brave New World, I will re-read those, it's been 20 years or more since I last read those

Here is a picture of some of these books.....

Here is the full list, the titles link to Amazon so you can read the reviews there

Hyperion by Dan Simmons
The Fall of Hyperion by Dan Simmons
The Rise of Endymion by Dan Simmons
Endymion by Dan Simmons
Philip and Alexander Kings and Conquerors by Adrian Goldsworthy
The Prince by Macchiavelli
The Dutch Overseas Empire by Pieter C. Emmer, Jos J.L. Gommans
1776: The riveting story of George Washington by David McCullough
The Great Bridge by David McCullough
Ulysses by James Joyce
The Fifth Risk: Undoing Democracy by Michael Lewis
The Andromeda Strain by Michael Crichton
A Tree Grows in Brooklyn by Betty Smith
The Killer Angels: The Classic Novel of the Civil War Paperback by Michael Shaara
The Red Queen: Sex and the Evolution of Human Nature by Matt Ridley
How Computers Really Work A Hands-On Guide to the Inner Workings of the Machine by Matthew Justice
Can't Slow Down: How 1984 Became Pop's Blockbuster Year by Michaelangelo Matos
Powers of Darkness: The Lost Version of Dracula by Bram Stoker , Valdimar Ásmundsson, et al
Black Box Thinking The Surprising Truth About Success by Matthew Syed
Life on Mars: What to Know Before We Go by David Weintraub
Brave New World by Aldous Huxley
Consider the Lobster: And Other Essays by David Foster Wallace
Pirate Latitudes by Michael Crichton
Infinite Jest by David Foster Wallace
100 Things Millionaires Do Little Lessons in Creating Wealth by Nigel Cumberland
Turing's Cathedral: The Origins of the Digital Universe by George Dyson
Where the Past Begins: Memory and Imagination by Amy Tan
Gerald's Game by Stephen King
If It Bleeds by Stephen King
The Eyes of the Dragon by Stephen King
The Dead Zone by Stephen King
The Running Man by Stephen King
Graveyard Book by Neil Gaiman
Masters of Doom: How Two Guys Created an Empire and Transformed Pop Culture by David Kushner
SQL Server 2019 Revealed Including Big Data Clusters and Machine Learning by Bob Ward
The Man In The High Castle by Philip K. Dick

18,300 pushups

Do about 100 pushups every other day.. this adds up to about 18,300 pushups for the whole year

10,000 in dividends

Last year I received 9551 in dividends, since companies increase the dividend payout usually every 4 quarters and since I reinvest them, I should be easily able to get 10K

300 Hours of Pluralsight watching

I plan to watch about 6 hours a week of Pluralsight content. About 1 hour at lunch time and then 1 hour over the weekend

Get 10 of these book in 2021

Get at least 10 9 books of the "wish list" below

I have heard about these from friends or family. I also might have heard about these books from the various podcasts I listen to

The Splendid and the Vile: A Saga of Churchill, Family and Defiance During the Blitz by Eric Larson
Salt, Fat, Acid, Heat: Mastering the Elements of Good Cooking by Samin Nosrat
The Lessons of History by Will Durant and Ariel Durant
1/1 ~~The Prince by Macchiavelli~~
The Great War: A Combat History of the First World War by Peter Hart
Hero with a 1000 faces by Joseph Campbell
The Virtues of War: A Novel of Alexander the Great by Steven Pressfield
Persian Fire: The First World Empire and the Battle for the West by Tom Holland
Blood and Thunder by Hampton Sides
Empire of the Summer Moon: Quanah Parker and the Rise and Fall of the Comanches, the Most Powerful Indian Tribe in American History by S. C. Gwynne
The Storm Before the Storm: The Beginning of the End of the Roman Republic by Mike Duncan
Black Elk: The Life of an American Visionary by Joe Jackson
What I Talk About When I Talk About Running: A Memoir by Haruki Murakami
The Long Ships by Frans G. Bengtsson
A Gentleman in Moscow by Amor Towles
The Innocent Anthropologist : Notes from a Mud Hut by Nigel Barley
Shogun: The Epic Novel of Japan by James Clavell
Cat's Cradle by Kurt Vonnegut
Zorba the Greek by Nikos Kazantzakis
The Rising Sun: The Decline and Fall of the Japanese Empire, 1936-1945 by John Toland
Shattered Sword: The Untold Story of the Battle of Midway by Jonathan Parshall, Anthony Tully
The Road by Cormac McCarthy
The End of October: A Novel by Lawrence Wright
Change Agent by Daniel Suarez
Darkness at Noon by Arthur Koestler
A Distant Mirror: The Calamitous Fourteenth Century by Barbara Tuchman
Azure SQL Revealed: A Guide to the Cloud for SQL Server Professionals by Bob Ward

Plant all these peppers

We got the following seeds in December

The 3 packs on the left side are the Carolina Reaper, Trinidad Scorpion and Apocalypse Scorpion. These are some of the hottest peppers.... these things just look menacing, you can see they look spicy

Last year we planted 31 plants, you can those in the 2020 in Numbers... post

I am excited about the chocolate habaneros, those look pretty cool

Progress so far

Activity	Count	%
Pushups	370	2.02%
Run	22.17	2.77%
Pluralsight Hours Watched	2.13	0.711%
Books Read	1	2%
Dividends	$63.89	0.6389%

Done

Book Finished

Date Title

----- -------------------------------------------------------

Jan 3 St Petersburg

Pluralsight Course Finished

Date Title

----- -------------------------------------------------------

Jan 6 Optimizing Query Performance with Columnstore Indexes

2020 in Numbers...

2021-01-02T11:50:00.002-08:00

I am glad 2020 ended and hopefully 2021 will be much better. A couple of items on my list for this year got either postponed or will never happen. I planned to go to the PASS Summit in Texas in 2020, that obviously didn't happen and it looks like it won't happen in the future either. PASS is no more :-(

Here are some numbers for 2020 that I put together looking at some sheets and journals

123,116
Calories burned running. I ran a little over 800 miles this year and burned what seems like a lot of calories. But since we are all home, the wife and kids did a lot of baking, this evened out the calories :-)

13,930
Pushups. I try to do 100 pushups every 2 or 3 days, this is the total for the year

9,551
Dividends. This is the total amount of dividends I received this year. All of these have been reinvested. Until I retire, I am doing a reinvestment into the same security

800.3
Miles run. My goal was 800 and I stuck to it... It's nothing major, about 4 miles 4 times a week

300

Days since I was last in the office

I told my wife when the office is open again, you won't see me for a month :-)

60
The number of books I read

Since there is no commute, I had some more time to read this year, I read about 5 books per month

Some of the book I read

Will have my 5 favorite books after the numbers list

Longest bike ride.

Did this bike ride with my oldest son. We left around 6:40 AM because it went up to 95 degrees that day. I Also managed to get a flat tire 30 miles in :-(

After we came home we just sat on the couch for the rest of the day

Number of pepper plants planted

We planted 31 spicy pepper plants. We picked a lot of the peppers, gave away a bunch, made pepper flakes etc etc

Here are some pics

The first one is the ghost pepper

Seasons of shows I watched and finished with my wife

Narcos Mexico II

Mr Robot I

Mr Robot II

Mr Robot III

Mr Robot IV

The Crown I

Succession I

Succession II

Succession III

Ozark III

Curb Your Enthusiasm X

Number of tables I dropped on the production database server

Best item purchased

Since I am working from home constantly, I decided to invest in a universal docking station. We got new laptops from work, they only come with 2 USB ports, and of course these are right next to the HDMI port. Using this USB docking station makes live so much easier

The one I picked is the Hiearcool USB C Hub,USB-C Laptop Docking Station,11 in 1 Triple Display Type C Adapter Compatiable for MacBook and Windows(2HDMI VGA PD3.0 SD TF Card Reader Gigabit Ethernet 4USB Ports)

And here is what it looks like, I included a mouse for scale (sorry didn't have a banana)

As you can see it comes with a nice pouch, it's really small and fits easily in your bag.

Best 5 books

Here are my favorite 5 books I read last year, I did not include books I re-read like The Stand and Fellowship of the ring

Sandworm: A New Era of Cyberwar and the Hunt for the Kremlin's Most Dangerous Hackers by Andy Greenberg

This is even more relevant now after the SolarWind hack

Alexander Hamilton by Ron Chernow

I admit I did not know much about Hamilton, this was recommended by my brother in law. This is a fantastic book about a man who was part of the beginnings of the US

The Terror by Dan Simmons

I watched the show 2 years ago on AMC, the show is really good... but the book is even better

Permutation City by Greg Egan

Will this be us in 100 years?

The Wizard and the Prophet: Two Remarkable Scientists and Their Dueling Visions to Shape Tomorrow's World by Charles Mann

There are two ways to go about things... either conserve.. or use technology to fix things.....

That's it for 2020...... I will be back in a couple of days to show my list of things for 2021....

Have a good 2021

TWID May 16, 2020: random potpourri of things

2020-05-15T12:30:00.001-07:00

In the back of the house we have transferred the plants to a raised bed

It is over 80 degrees today, hopefully the plants will love this weather and grow big

For Mothers day... we didn't do much because we are still under lockdown.. my oldest son made this crumb cake.. it was delicious

This Week I Tweeted

US video game sales have record quarter, as consumers stay at home

New numbers from NPD confirm what we’ve known for a while: The first quarter of 2020 was a very good one for gaming companies. The new report notes that sales hit a record $10.86 billion in the States between January and March of this year, marking a 9% increase over a year prior; $9.58 billion of that figure was from video game content.

The primary driver is, you guessed it, COVID-19. As stay at home orders have been enacted on the federal and state levels, people are coping with the ongoing daily horror that is life in 2020 by playing video games. Lots and lots of video games.

Some cool stuff you might enjoy

Most detailed ever photograph of The Night Watch goes online

The Rijksmuseum is today publishing the largest and most detailed ever photograph of The Night Watch on its website, making it possible to zoom in on individual brushstrokes and even particles of pigment in the painting. Work on Operation Night Watch will resume on Wednesday 13 May in the glass chamber in the museum.

I have seen this painting many times, the reason is because I grew up in Amsterdam and my neighbor worked for the Rijksmuseum. I always got complementary tickets. The interesting part is that I have been to the Rijksmuseum more times when moving to the US than I did when I lived in Amsterdam :-(

This does looks pretty cool, you can just keep zooming. Direct link http://hyper-resolution.org/Nightwatch

Why millennials have fallen in love with stamp collecting

Covid has increased the number of people getting interested in stamp collecting. Here are 3 articles I happen to see in the past couple of weeks

Post modern: why millennials have fallen in love with stamp collecting

Philately is gaining popularity with younger hobbyists, who are drawn to its vintage - and Instagram - appeal

It was once a byword for boring, but it seems stamp collecting is finding a new – young – audience, despite the fact that some hobbyists have never actually posted a letter.

Philately is gaining popularity among millennials, many of whom see the creative pursuit as an escape from their screen-based lives, says 37-year-old Suzanne Rae, from North Yorkshire.

“Philately is tangible: it’s relaxing and unplugged. It’s also very Instagrammable,” says Rae, who is only the second female chairman the Philatelic Traders Society has had in its 91-year history. “Twitter and Instagram enable young collectors to find people like them, and see that it’s not only a geeky old man’s pursuit.”

Covid-19 Virus Affect On The Stamp Market

The strong reaction of the US Government to the Covid pandemic has resulted in the self-isolation and self-quarantining of millions of Americans as well as people worldwide. While this is having a devastating effect on the economy, it has been a real boost to the stamp market. Stamp collectors around the world suddenly have many hours of time to fill since they can't go out, go to work or watch sporting events. Hence, they spend time buying stamps. The dealers I spoke with all report a surge in orders, many from customers they have never had before.

COVID-19 a shot in the arm for stamp collecting

Well, if you’d asked me a few months back what was needed to give stamp collecting a kick up the backside, my first answer would probably not have been “a global supervirus pandemic.”

But it sure feels like that’s what has happened. The enforced worldwide recreation break has existing collectors burying their heads into their albums, lapsed collectors dusting cobwebs off old shoeboxes, and curious novices taking their first tentative steps into a mysterious world.

New philatelic blogs and Instagram feeds are starting up, and online viewers are devouring YouTube clips. Philatelic magazines have even been offering free access to an edition or two.

Even the mainstream media have noticed. Stamp dealers report rising sales. Kids are writing letters. Even before the COVID-19 crisis kicked in, both the BBC and the Observer were reporting the growing popularity of stamp collecting among younger collectors.

Some Progress

Books finished so far/in progress

In progress

Gotham: A History of New York City to 1898 84% done

The History of the Ancient World: From the Earliest Accounts to the Fall of Rome 44% done

Return Of The King 4% done

Done with these

Book	Completed
Stealth War: How China Took Over While America's Elite Slept	01/04
Sandworm: A New Era of Cyberwar and the Hunt for the Kremlin's Most Dangerous Hackers	01/25
Alexander Hamilton	02/12
The Quick Python Book, 3rd Edition	02/13
The Gene: An intimate history	02/17
Moonwalking with Einstein	02/27
The Somme	02/25
The Art of Learning	02/16
Ready to Run: Unlocking Your Potential to Run Naturally	03/08
The Tudors, a very short introduction	03/17
South: The Illustrated Story of Shackleton's Last Expedition 1914-1917	03/18
The Body: A Guide for Occupants	03/30
The Great Sea: A Human History of the Mediterranean	04/07
Tiny Habits: The Small Changes That Change Everything	04/11
Lifespan: Why We Age – and Why We Don’t Have To	04/27
The Stand	04/30
Fellowship of the ring	05/07
The Two Towers	05/15

Books in the queue

These are next... in no specific order

Books to get
Here are the books I plan to get. These were either mentioned on more than one podcast, recommended by friends, or are new books from authors I like, for example Eric Larson's latest book about Churchill and the Blitz

The Splendid and the Vile: A Saga of Churchill, Family and Defiance During the Blitz by Eric Larson
Ulysses by James Joyce
Salt, Fat, Acid, Heat: Mastering the Elements of Good Cooking by Samin Nosrat
The Lessons of History by Will Durant and Ariel Durant
The Prince by Macchiavelli
The Great War: A Combat History of the First World War by Peter Hart
Hero with a 1000 faces by Joseph Campbell
The Virtues of War: A Novel of Alexander the Great by Steven Pressfield
Persian Fire: The First World Empire and the Battle for the West by Tom Holland
Blood and Thunder by Hampton Sides
Empire of the Summer Moon: Quanah Parker and the Rise and Fall of the Comanches, the Most Powerful Indian Tribe in American History by S. C. Gwynne
The Storm Before the Storm: The Beginning of the End of the Roman Republic by Mike Duncan
Permutation City by Greg Egan
Infinite Jest by David Foster Wallace
Black Elk: The Life of an American Visionary by Joe Jackson
What I Talk About When I Talk About Running: A Memoir by Haruki Murakami
The Long Ships by Frans G. Bengtsson
A Gentleman in Moscow by Amor Towles
The Innocent Anthropologist : Notes from a Mud Hut by Nigel Barley
Shogun: The Epic Novel of Japan by James Clavell
Cat's Cradle by Kurt Vonnegut
Zorba the Greek by Nikos Kazantzakis

Running progress
My goal is to run 800 miles this year, this means about 4 runs of about 4 miles per week

Miles Percentage
296.81 37.10%

So far that burned a bit under 45,600 calories

TWID May 9, 2020: Books and Plants

2020-05-09T06:36:00.001-07:00

This is a post detailing some stuff I did, learned, posted and tweeted this week, I call this TWID (This week in Denis). I am doing this mostly for myself... a kind of an online journal so that I can look back on this later on. Will use the label TWID for these

It has been a while since I posted here.. but I decided to bring back the TWID posts again

I have been reading a lot lately, since I am already home for 60 days, I decided to read some of the big books I own or re-read them. I already finished the Stand, almost done with Gotham and I am 35% done with the Lord of The Rings book.

I read 10 pages per day of the Gotham book and then I usually have a book I read at home and one while I commute. Since I am home now, I read the commuting book in the morning

Here is a pic of the books I owe but never read (except for the Lord Of the Rings book)

At the end of this post, you can see all the books I finished, what I am reading now and also what is in my backlog/queue to read.

Here you can see some of the seeds my son and I purchased

That's right.. the second one from the left in the top is a ghost chile... not sure yet what I will do with those. Maybe make pepper spray or some extremely hot sauce :-)

This is what the plants looked like a couple of weeks ago

This is what it looks like today

As you can see they are ready to be planted outside. It still cold at night but after Wednesday May 12th, these should be fine to be transplanted

Here you can see what the plants look like 2-3 weeks ago with a fan blowing on them

Put a fan blowing on these to prepare for the harsh outside world when they go outside around Mother’s Day #gardening #tomatoes #peppers #ghostpepper pic.twitter.com/BT5UcG5oAI
— Denis Gobo (@DenisGobo) April 25, 2020

The reason you use a fan inside is to strengthen the the plants, this way they don't break when you put them outside and it is a little windy on the first day

This Week I Tweeted

One Trader Started The Day With $77,000 In His Account; By The End He Owed $9 Million

As a result, crude was actually trading at a negative $3.70 a barrel when Shah’s screen had it at 1 cent; the reason: Interactive Brokers never displayed a subzero price to him as oil kept diving to end the day at minus $37.63 a barrel.
At midnight, Shah some very bad news: he owed Interactive Brokers $9 million. He’d started the day with $77,000 in his account, expecting that his biggest possible loss was 100%, or $77,000.

Ouch that is so bad, who knew oil could go negative....

Can't quit Excel

This happened to me.. pasted a lot of data into Excel on a VM and then tried to cancel it and quit.... nope. So I just used task manager to kill it

Some cool stuff you might enjoy

You know about waitfor delay but did you know there is a waitfor time?

You may or may not know that WAITFOR has TIME in addition to DELAY

Did you know, I sold kinderzegels while in 6th grade?

When I sold kinderzegels as part of school (semipostal stamps that benefit child welfare)

Mindmap Diagram Codes

This is pretty cool, you can create a mindmap, code on left.. output on the right, you can share it as well

Looks like this

Some Progress

Books finished so far/in progress

In progress

Gotham: A History of New York City to 1898 78% done

The History of the Ancient World: From the Earliest Accounts to the Fall of Rome 20% done

The Two Towers 21% done

Done with these

Book	Completed
Stealth War: How China Took Over While America's Elite Slept	01/04
Sandworm: A New Era of Cyberwar and the Hunt for the Kremlin's Most Dangerous Hackers	01/25
Alexander Hamilton	02/12
The Quick Python Book, 3rd Edition	02/13
The Gene: An intimate history	02/17
Moonwalking with Einstein	02/27
The Somme	02/25
The Art of Learning	02/16
Ready to Run: Unlocking Your Potential to Run Naturally	03/08
The Tudors, a very short introduction	03/17
South: The Illustrated Story of Shackleton's Last Expedition 1914-1917	03/18
The Body: A Guide for Occupants	03/30
The Great Sea: A Human History of the Mediterranean	04/07
Tiny Habits: The Small Changes That Change Everything	04/11
Lifespan: Why We Age – and Why We Don’t Have To	04/27
The Stand	04/30
Fellowship of the ring	05/07

Books in the queue

These are next... in no specific order

Books to get
Here are the books I plan to get. These were either mentioned on more than one podcast, recommended by friends or are new books from authors I like, for example Eric Larson's latest book about Churchill and the Blitz

The Splendid and the Vile: A Saga of Churchill, Family and Defiance During the Blitz by Eric Larson
Ulysses by James Joyce
Salt, Fat, Acid, Heat: Mastering the Elements of Good Cooking by Samin Nosrat
The Lessons of History by Will Durant and Ariel Durant
The Prince by Macchiavelli
The Great War: A Combat History of the First World War by Peter Hart
Hero with a 1000 faces by Joseph Campbell
The Virtues of War: A Novel of Alexander the Great by Steven Pressfield
Persian Fire: The First World Empire and the Battle for the West by Tom Holland
Blood and Thunder by Hampton Sides
Empire of the Summer Moon: Quanah Parker and the Rise and Fall of the Comanches, the Most Powerful Indian Tribe in American History by S. C. Gwynne
The Storm Before the Storm: The Beginning of the End of the Roman Republic by Mike Duncan
Permutation City by Greg Egan
Infinite Jest by David Foster Wallace
Black Elk: The Life of an American Visionary by Joe Jackson
What I Talk About When I Talk About Running: A Memoir by Haruki Murakami
The Long Ships by Frans G. Bengtsson
A Gentleman in Moscow by Amor Towles
The Innocent Anthropologist : Notes from a Mud Hut by Nigel Barley
Shogun: The Epic Novel of Japan by James Clavell
Cat's Cradle by Kurt Vonnegut
Zorba the Greek by Nikos Kazantzakis

Running progress
My goal is to run 800 miles this year, this means about 4 runs of about 4 miles per week

Miles Percentage
279.57 34.95%

So far that burned a bit under 43 thousand calories

Interesting quote....heard it on a podcast... don't remember which one .. sorry :-(

A fool never learns

A smart man learns from his mistakes

A wise man learns from the mistakes of others

A quick and easy way to count the percentage of nulls without a where clause in PostgreSQL

2019-04-30T10:21:00.000-07:00

This came up the other day, someone wanted to know the percentage of NULL values in a column

Then I said "I bet you I can run that query without using a NULL in the WHERE clause, as a matter of fact, I can run that query without a WHERE clause at all!!"

I created a SQL Server version of this post here: How to count NULLS without using IS NULL in a WHERE clause, this is the PostgreSQL version

To start, first create this table and verify you have 9 rows

CREATE TABLE foo(bar int);
INSERT INTO foo values(1),(null),(2),(3),(4),
 (null),(5),(6),(7);

SELECT * FROM foo;

Here is what the output should look like

To get the NULL values and NON NULL values, you can do something like this

SELECT COUNT(*) as CountAll FROM foo WHERE bar IS NOT NULL;
SELECT COUNT(*) as CountAll FROM foo WHERE bar IS  NULL;

However, there is another way

Did you know that COUNT behaves differently if you use a column name compared to when you use *

Take a look

SELECT COUNT(*) as CountAll, 
  COUNT(bar) as CountColumn
FROM foo;

If you ran that query, the result is the following

CountAll CountColumn
----------- -----------
9 7

Here is what it looks like in pgAdmin3

Let's see what the PostgreSQL documentation has to say

COUNT(*) number of input rows

COUNT(expression) number of input rows for which the value of expression is not null

This is thus indeed documented behavior, count(*) counts all rows, count(column) skips the rows with NULLS

So now, lets change our query to return the percentage of non null values in the column

Here is what the query looks like

SELECT COUNT(*) as CountAll, 
  COUNT(bar) as CountColumn, 
  (COUNT(bar)/COUNT(*))*100 as PercentageOfNonNullValues 
FROM foo;

Running that gives us this output

CountAll CountColumn percentageOfNonNullValues
----------- ----------- ---------------------------------------
9 7 0

Here is what it looks like in pgAdmin3

As you can see, we have a 0 in the percentage column. The reason is that if you do math with integers, you will get an integer back

We can fix this easy by multiplying one of the integers by 1.0, this will convert the integer to a numeric data type . So instead of this line

 (COUNT(bar)/COUNT(*))*100 as PercentageOfNonNullValues

We will change it to be like this

(COUNT(bar)*1.0/COUNT(*))*100 as PercentageOfNonNullValues

Here is the changed query

SELECT COUNT(*) as CountAll, 
  COUNT(bar) as CountColumn, 
  (COUNT(bar)*1.0/COUNT(*))*100 as PercentageOfNonNullValues 
FROM foo;

Here is the output

CountAll CountColumn percentageOfNonNullValues
----------- ----------- ---------------------------------------
9 7 77.7777777777800

Here is what it looks like in pgAdmin3

Instead of doing * 1.0, you can also explicitly cast the integer like this

(CAST (COUNT(bar) as numeric(20,10))/COUNT(*))*100 as PercentageOfNonNullValues

The query now looks like this

SELECT COUNT(*) as CountAll, 
  COUNT(bar) as CountColumn, 
  (CAST (COUNT(bar) as numeric(20,10))/COUNT(*))*100 as PercentageOfNonNullValues
FROM foo;

Here is what it looks like in pgAdmin3

That's it for this short post.. hopefully you knew this, if not, then you know it now :-)

The Vessel, a honeycomb shaped structure that will be a great tourist destination at Hudson Yards in New York City

2019-04-17T07:27:00.001-07:00

I took a stroll at the high-line in NYC yesterday and noticed that the Vessel was open to the public

The Vessel is a copper-colored steel structure comprises 54 interconnecting flights of stairs, 2,500 steps, and 80 landings, it was designed by Heatherwick Studio's. The Vessel rises to a height of 150 ft (45 m) offering views of the city and the Hudson River.

Here is what it looks like

Here it is on the map, it's right at the end/start of the highline and also just a block or so from the Jacob K. Javits Convention Center. I think a lot of visitors to the Jacob Javits Center will stop by

The first thing you will notice is how shiny the Vessel is... in place when the sun hits it..it's like hitting a mirror. I wonder how soon before it turns green because of oxidation. I wouldn't want to be the person who has to polish this copper behemoth

The second thing you need to know is that you can't go inside the Vessel without a ticket. Tickets are free, I picked one up in front of the Vessel near the building right in front of it. I did have to wait 30 minutes to get in, the tickets are only valid for 1 hour

The third thing you will notice once you enter the Vessel is a blue light and a crowd of people near it

You will see something like this

Or something like this

That spot with the blue light is the selfie station, you put your phone down in the middle, then you stand there while a selfie is taken

I decided not to do that since I don't take selfies, but here is the view from the bottom when you take a normal shot

Here you can see all the stairs, it's nice to walk all over the structure, you get nice views of the city.

If for some reason, you can't take the stairs, there is an elevator as well. The thing you see in the middle in the pic below is where the elevator travels

Here is a pic I took of people on the top of the Vessel, it was very windy yesterday, I stayed on the one below the top

I took a picture of these path trains

TWID March 30, 2019: Tsundoku, DevOps, robots, turing award,gps drifting

2019-03-30T10:05:00.001-07:00

This Week I Learned

Tsundoku
Tsundoku (積ん読) is acquiring reading materials but letting them pile up in one's home without reading them. The term originated in the Meiji era (1868–1912) as Japanese slang. It combines elements of tsunde-oku (積んでおく, to pile things up ready for later and leave) and dokusho (読書, reading books). It is also used to refer to books ready for reading later when they are on a bookshelf. As currently written, the word combines the characters for "pile up" (積) and the character for "read" (読).

OK, so I have a bunch of books that I bought years ago and never finished. Some of these I never started. So what I decided to do is take 1 hour each day before going to work, grab 3 books and read 20 minutes per book. I also have some books in the office, I am reading 20 minutes each time when I go to the office, this is currently twice a week. The book I am reading in the office is Pragmatic Thinking and Learning, I am 40% done with that book

I already finished the book War Of Art, am 25% done with Tools of Titans and just started Python Tricks

Here is a pic of the books

The books in that picture are the following

Tools of Titans: The Tactics, Routines, and Habits of Billionaires, Icons, and World-Class Performers (started)
Tribe of Mentors: Short Life Advice from the Best in the World
War Of Art (finished)
Wired To Eat
Python Tricks the book: A Buffet of Awesome Python Features (started)
Fluent Python: Clear, Concise, and Effective Programming
Book of Five Rings
How Linux Works, 2nd Edition: What Every Superuser Should Know

I continued my DevOps training and like last week, I decided to put my note at the end of this post, this way I can easily look it up again

This Week I Tweeted

Three ‘Godfathers of Deep Learning’ Selected for Turing Award

Three computer scientists who laid the foundations for many of the recent advances in artificial intelligence are being honored with this year’s Turing Award, considered the field’s highest accolade.

Geoff Hinton, an emeritus professor at the University of Toronto and a senior researcher at Alphabet Inc.’s Google Brain, Yann LeCun, a professor at New York University and the chief AI scientist at Facebook Inc., and Yoshua Bengio, a professor at the University of Montreal as well as co-founder of AI company Element AI Inc., will share this year’s award, which is given annually by the Association for Computing Machinery.

The three winners will split a $1 million prize that comes with the award, which is currently underwritten by Google

For a list of previous winners see here: https://en.wikipedia.org/wiki/Turing_Awardhttps://en.wikipedia.org/wiki/Turing_Award

Was MongoDB Ever the Right Choice?

I was reading a post recently about Red Hat removing MongoDB support from Satellite (and yes, some folks say it is because of the license changes). It made me think how often over the last few years I’ve seen post after angry post about how terrible MongoDB is and how no one should ever use. However, in all this time, MongoDB has become a much more mature product. So what happened? Did all of the hate truly come from mistakes made in the early implementation/marketing of MongoDB? Or is the problem that people are blaming MongoDB for their own lack of efforts when evaluating if it was a good fit?

Read and find out.....

Australia Is Drifting So Fast GPS Can't Keep Up

Australia is not quite where you think it is. The continent has shifted by 4.9 feet since the last adjustment was made to GPS coordinates in 1994, reports the New York Times.

A significant correction must be made by the end of the year for navigation technology to keep working smoothly.

This is interesting, I about drift because undersea cables will break, but never had an idea that it was this fast that they have to adjust GPS

Boston Dynamics’ new robot stacks boxes

It’s a “mobile manipulation robot” designed for the logistics sector. It can autonomously stack and unstack boxes onto and off pallets, and shift them onto conveyor belts. It uses an onboard vision system to track which objects go where, and to judge how to grasp and place each box. It uses a robotic technique called “force control” to nestle each box up against its neighbors. It can handle weights of up to 15 kilograms (33 pounds.)

And that is how it all starts... wondering when they will rename the company to SkyNet :-)

Some cool stuff you might enjoy

Awesome Newsletters
A curated list of awesome newsletters.

Inspired by the awesome-* trend on GitHub.

There is something here for everyone

The most populous cities in the world from 1500 to 2018

This is amazing, make sure to watch the whole animation..it's like a race....

Some DevOps notes

This is for me.. not for you.... but feel free to take a look...although it won't probably make a lot of sense if you see these terms for the first time

Top 10 books from 10 to number 1
Visible Ops
Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation
Release It!: Design and Deploy Production-Ready Software
Effective DevOps: Building a Culture of Collaboration, Affinity, and Tooling at Scale
Lean Software Development: An Agile Toolkit
Web Operations: Keeping the Data on Time
The Practice of Cloud System Administration: DevOps and SRE Practices for Web Services
The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations
Leading the Transformation: Applying Agile and DevOps Principles at Scale
The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win

Devopsweekly.com mailinglist

Wall of Confusion
Impedance mismatch between teams

Blameless postmortems
---------------------------------------
It's a meeting
within 48 hours of incident
run by a 3rd party

Have everything in a timeline. Not here to blame a person but making sure this doesn't happen again

Trasnparent Uptime
-------------------------
Admit failure
Sound like a human.. no doublespeak
Have a communication channel
Be authentic

Trust blockers
--------------------
Lack of Context
Conflicting goals

Open it Up
-----------------------
Chat rooms
Wiki pages
Source code (to read)
Infrastructure
Monitoring Tools
Ticket Tracker

Chatops
USe something like chat or teams instead of emails

Shared responsibility
-------------------------------
Feedback
Automation
Team culture
No Silos

Shadow IT/Bimodal IT
Groups deliberate bypassing processes to get things done

Devs responsibe for failed deployment after code checkin

Changing your ways of how you do work.... might be hard..but it will pay off

Conway's law
----------------------------
organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations.

Eric S. Raymond, an open-source advocate, restated Conway's law in The New Hacker's Dictionary
James O. Coplien and Neil B. Harrison stated:

If the parts of an organization (e.g., teams, departments, or subdivisions) do not closely reflect the essential parts of the product, or if the relationship between organizations do not reflect the relationships between product parts, then the project will be in trouble ... Therefore: Make sure the organization is compatible with the product architecture.

Westrum Model
--------------------------------
Pathological (Power Oriented)
Bureaucratic (Rule Oriented)
Generative (Performance Oriented)

Management Best Practices
-----------------------------------
Independent, cross-functional teams
People first
Agile, lean processes

Kaizen
---------------------------
Small change aka continues improvement
Good processes bring good results
Gemba (Go see for yourself).. (the real pace or Locus).. meaning look at the code or the system or factory floor
Speak with data, manage by facts (scientific method)
Take action to contain and correct root causes (The 5 whys, people don't fail processes do)
Work as a team
Kaizen is everubody's business

======================================================
------------------------------------------------------
======================================================

Waterfall --> Iterative
DevOps has strong roots in Agile but is not just agile

Lean, 7 principles
1 Eliminate Waste
2 Amplify Learning
3 Decide as late as possible
4 Decide as fast as possible
5 Empower the team
6 Build in integrity
7 See the whole picture

Waste
Muda: work that absorbs resources but adds no value
Muri: Unreasonable work imposed on workers and machines
Mura: Work coming in unevenly instead of a constant or regular flow

The 7 Wastes Of Software
1 Partially work done
2 Extar features
3 Relearning
4 Handoffs
5 Delays
6 Task Switching
7 Defects

Build Measure Learn
-----------------------
Build minimum viable product
Measure the outcome and internal metrics
Learn about you problem and your solution
Repeat, go deep where it's needed

Concept to cash: from idea to it's realization, including everything needed to get it to customer

ITSM (Information Technology Service Management)

ITIL (Information Technology Infrastructure Library)
1 Service Strategy
2 Service Design
3 Service Transition
4 Service Operation

ITIL 2007... 2000 pages long... The Visible OPS handbook (only 100 pages)

Use ITIL but make it lean

======================================================
------------------------------------------------------
======================================================

Infrastructure as code
Everything programmatically, no UI, same code runs on test, stage, UAT and production

Servers are cattle not pets.. deploy in mass, not handcrafted

Provisioning: Making a server ready for operation..including OS, network connectivity and system services
Deployment: Automatically deploying and upgrading applications on a server
Orchestration: Performing coordinated operations across multiple systems
Configuration Management: Management of change control after initial provision, maintaining and upgrading apps and app dependencies

Imperative/procedural: Commands necessary to produce a desired state are defined and executed
Declaritive/Functional: A desired state is defined, relying on the tool to configure a system to match that state
Idempotent: The ability to execute repeatedly, resulting in the same outcome
Self Service: The ability for an end user to initiate a process without having to go through other people

Canary (staged) Deployment Pattern
Upgrade 1 server, see how it works before upgrading the rest

Blue Green deployment
Two environment, code is pushed to one, then environments are swapped, variant like cluster immune system deployment

Containers making things easier to deploy, golden image is coming back now since you can just packe it up as a container.

Terraform.. used so that you can use setting across Azure and AWS

---------------------------------------------------------------

Continuous Deploy
Continuous Delivery
Continuous Integration

Time to market goes down... high performing IT orgs can deploy on demand, some 10 or 100 times a day

“Cease dependence on inspection to achieve quality. Eliminate the need for massive inspection by building quality into the product in the first place.” quote by W. Edwards Deming

With Continuous Delivery you can much easier see what change caused a degradation of an application, with a quarterly release there are so many changes, it is difficult to find out what caused the degradation

The goal of continuous integration is that working software is in a working state all the time quote by Jez Humble

6 practices
Builds should pass the coffee test (should be less than 5 minutes)
Commit really small bits
Don't leave the build broken
Use a trunk based development flow
No crappy tests, fix your tests
The build should return a status, a log and an artifact

Only build artifacts once..immutable, should not change... this creates trust.. underlying bits did not change
Stop deploy if a step fails, notify teams

Test Unit testing
Code hygiene linters, formatters, banned functions check
Integration testing

TDD/BDD/ATDD

TDD
State desired outcome as test
write code to pass test
Repeat

BDD
Work with stakeholders
Describe business functionality
Tests are based on natutal language descriptions

ATDD (acceptance test driven development)
End user perpective
Use case automated testing
testing is continous during development

3 ways to get around slow tests... so you don't violate your coffee rule
use nonblocking tests
use time scheduled tests
use monitoring

MTTR
Mean Time To Recovery

Cascading failure pattern: Cascading failure is number one threat to stability, one layer can choke out other layers
Circuit breaker pattern: If app see an issue with one specific set of calls redirect to other part.. minimizes outage risk

The twelve-factor methodology
https://12factor.net/

I. Codebase
One codebase tracked in revision control, many deploys

II. Dependencies
Explicitly declare and isolate dependencies

III. Config
Store config in the environment

IV. Backing services
Treat backing services as attached resources

V. Build, release, run
Strictly separate build and run stages

VI. Processes
Execute the app as one or more stateless processes

VII. Port binding
Export services via port binding

VIII. Concurrency
Scale out via the process model

IX. Disposability
Maximize robustness with fast startup and graceful shutdown

X. Dev/prod parity
Keep development, staging, and production as similar as possible

XI. Logs
Treat logs as event streams

XII. Admin processes
Run admin/management tasks as one-off processes

If it hurts, do it more often. The best way to avoid failure is to fail constantly

Monitoring
------------------
Service performance/uptime
software component metrics
system metrics
app metrics
performance
security

5 Principles of log data
Do not collect log data if you never plan to use it
Retain log data for as long as it can be used
Log all you can, alert only what you must respond to, define log levels
Don't make logging more available than production stack
Make log changes as needed

Future
Containers and Serverless (serverless might only run 1 invocation)

Security
100:10:1 (Dev:Ops:Sec)
100 developers, 10 Ops, 1 Security person

The Rugged Manifesto: https://ruggedsoftware.org/

I am rugged and, more importantly, my code is rugged.
I recognize that software has become a foundation of our modern world.
I recognize the awesome responsibility that comes with this foundational role.
I recognize that my code will be used in ways I cannot anticipate, in ways it was not designed, and for longer than it was ever intended.
I recognize that my code will be attacked by talented and persistent adversaries who threaten our physical, economic, and national security.
I recognize these things - and I choose to be rugged.
I am rugged because I refuse to be a source of vulnerability or weakness.
I am rugged because I assure my code will support its mission.
I am rugged because my code can face these challenges and persist in spite of them.
I am rugged, not because it is easy, but because it is necessary and I am up for the challenge.

TWID March 24, 2019: Gout, Music Biz, Space 2.0, Luke Starkiller, space angels, DevOps Notes

2019-03-24T07:38:00.000-07:00

This is a post detailing some stuff I did, learned, posted and tweeted this week, I call this TWID (This week in Denis). I am doing this mostly for myself... a kind of an online journal so that I can look back on this later on. Will use the label TWID for these

Didn't do a TWID post for a couple of weeks, will continue regularly from this week on. Had an issue with my foot... still not sure if I sprained my toe or if it was gout. Didn't run for 3 weeks but started to run again this week. It feels good to run again in nature. This morning I heard so many woodpeckers

This Week I Learned

Finished the book Space 2.0: How Private Spaceflight, a Resurgent NASA, and International Partners are Creating a New Space Age
Some interesting stuff, I like how the cost went down a lot after SpaceX, Blue Horizon and others made the government contractor

Started on the book Loonshots: How to Nurture the Crazy Ideas That Win Wars, Cure Diseases, and Transform Industries

Listened to the Tim Ferriss podcast with Safi Bahcall and in that podcast Safi mentioned that the 1st draft of Star Wars was so bad, no studio wanted it
It had these characters: Luke and Windy Starkiller. Lol can you imagine those names having made it into the movie

Summary of the Original Script of “The Star Wars”: https://www.starwarz.com/starkiller/summary-of-the-original-script-of-the-star-wars/

Listened tot the final episode of East Coast vs West Coats Business Wars

Learned some stuff about how it is changing how artists get paid in the music industry. Artists used to be paid by album, now they are paid by the song, 1500 streams is about $9, this is cut between label/produces/songwriter. Since artists are paid by song.. , songs are getting shorter and an album will have more songs. So if you listen through an album, you are listening to more songs. Artist are also putting the chorus at the start of the song to hook you in, because if you skip before 30 seconds or so, the play doesn't count.

Gompertz function
The Gompertz curve or Gompertz function, is a type of mathematical model for a time series and is named after Benjamin Gompertz (1779-1865). It is a sigmoid function which describes growth as being slowest at the start and end of a given time period

One example: Mobile phone uptake, where costs were initially high (so uptake was slow), followed by a period of rapid growth, followed by a slowing of uptake as saturation was reached.

The Fertility Doctor’s Secret
Her husband had given her a DNA test for Christmas because she was interested in genealogy. Her heritage turned out to be exactly what she had thought—Scottish, with English, Irish, and Scandinavian mixed in—and she never bothered to click on the link that would show whether anyone on the site shared her DNA.

Apparently she did have relatives on Ancestry.com—and not just distant cousins. The people now sending her messages said they were Cline’s secret biological children. They said their parents had also been treated by Cline. They said that decades ago, without ever telling his patients, Cline had used his own sperm to impregnate women who came to him for artificial insemination.

According to her DNA, Woock, too, was one of his children.

This so messed up, and with more and more people doing DNA testing, I think this will only increase.

But there is a person who has 600 offspring, his name is Bertold_Wiesner

Space Angels

Space Angels is Venture Capital firm, they invest in SpaceX as well as the ones in the list below and more

Accion Systems
Accion Systems is developing revolutionary ion beam propulsion technologies for satellites, that are light, powerful, and affordable.

Analytical Space
Analytical Space is developing a cost-effective, high-throughput satellite data relay service using Cubesats with laser downlink capabilities.

Astrobotic Technology
Astrobotic will be the first private company to regularly deliver customer payloads and communication services to the Moon's surface.

Atlas Space Operations
ATLAS' network of RF satellite ground stations will offer reliable delivery of big data from LEO satellites at one-third of the cost of legacy providers.

Because Learning
Because Learning is an interactive STEM platform that enables any school to run experiments from Earth to space, through the Spire satellite network.

Bridgesat
BridgeSat is developing an optical communications network that offers secure delivery of big data from LEO at low cost and high speeds.

NanoRacks
NanoRacks is the leading commercial provider of hardware and services in low-Earth orbit for microgravity research and space station utilization.

Took some DevOps trainging this week... wanted to save the notes I took.. so I put them at the end of this post :-)

This Week I Tweeted

The lawyers who took on Big Tobacco are aiming at Realtors and their 6% fee

Homeowners who are ready to sell their properties usually hire a real-estate agent to represent them by staging the home, photographing it, adding it to the MLS, marketing it, and showing it to prospective buyers. Sellers agree to pay that person a commission on the selling price of the home. That commission has traditionally been known as the “6%,” but it’s a little more complicated than that.

Sellers can really only negotiate with the agent they’ve hired, while agents representing buyers are generally assured of a standard 3% commission. That means that a seller’s agent who’s willing to negotiate, or one that works for a discount brokerage like Redfin RDFN, +2.93% , will be paid less than a buyer’s agent.

Buyers can choose to be represented by an agent, or to go without one – but in any case, all commission money for both sides of the deal is always paid by the seller, thanks to a 1996 NAR rule known as the “Buyer Broker Commission Rule.”

I always thought it was strange that listing a 100K house and a 2 Million house would give you so much more money for essentially the same amount of work. Of course selling a million dollar home takes more time. Then there are people who will talk to an agent...do the house tours and not put a bid down with that agent. If this prevails.. I wonder if the million dollar listing tv show will still be around?

Facebook Stored Hundreds of Millions of User Passwords in Plain Text for Years

Hundreds of millions of Facebook users had their account passwords stored in plain text and searchable by thousands of Facebook employees — in some cases going back to 2012, KrebsOnSecurity has learned. Facebook says an ongoing investigation has so far found no indication that employees have abused access to this data.

Facebook is probing a series of security failures in which employees built applications that logged unencrypted password data for Facebook users and stored it in plain text on internal company servers. That’s according to a senior Facebook employee who is familiar with the investigation and who spoke on condition of anonymity because they were not authorized to speak to the press.

This was maybe some logging, but still, this is bad

New Jersey becomes second state to ban cashless shops and restaurants

On Monday, New Jersey Governor Phil Murphy signed a bill banning cashless retail stores and restaurants in the Garden State. Murphy's signature makes New Jersey the second state in the US to ban cashless stores, after Massachusetts banned them in 1978.

More recently, New Jersey's move follows that of Philadelphia, which banned cashless stores earlier this month. Philadelphia's legislation was a reaction to a growing number of stores that only accept credit cards or require customers to pay with an app, like Amazon's new Amazon Go stores.

Ha, I ran into this the other day in Manhattan. Left the office with a $20 in my pocket, went to this place named Dos Torros with my co-worker, ordered a burrito and then couldn't pay, had to have my co-worker pay for me and then had to pay him back. My main reason yo pay with cash in restaurants and smaller shops is that I don't want my credit card to be hacked, my next reason is privacy.

Also some people with bad credit might now have credit cards.. how would they pay?

I welcome this law.

Some cool stuff you might enjoy

Ten Lessons I Learned While Teaching Myself to Code

The following is a guest post by Clive Thompson (@pomeranian99), a journalist who’s written about technology and science for two decades. Clive is a longtime contributing writer for the New York Times Magazine and a columnist for Wired.

In his guest post, Clive outlines the most important lessons he learned teaching himself to code after interviewing 200+ programmers for his new book Coders: The Making of a New Tribe and the Remaking of the World.

So, you want to learn to code.

Join the club! We live in a time when, as the venture capitalist Marc Andreessen famously put it, “…software is eating the world.” So the people who know how to program are in a catalytic spot; they can make things happen. Maybe you’ve watched this from the sidelines and thought: Huh. Could I learn to do that? Perhaps you’re out of school; maybe you can’t afford either the money or the time to go back and do a four-year degree in computer science. You’ve seen a zillion of these online tutorials in coding. Could you just sort of, well, teach yourself?

The short answer is: Sure you can.

The longer answer is… the rest of this essay.

Some interesting tidbits

Some numbers that you will know by heart if you have been working with SQL Server for a while

I was troubleshooting a deadlock the other day and it got me thinking.... I know the number 1205 by heart and know it is associated to a deadlock. What other numbers are there that you can associate to an event or object or limitation. For example 32767 will be known by a lot of people as the database id of the ResourceDb, master is 1, msdb is 4 etc etc.

A fun and quick posts with some numbers that most SQL Server peeps probably know

Some DevOps notes I took
Had some DevOps training, took some notes, didn't know where to save them, so put it here

CAMS
-------------
Culture
Automation
Measurement
Sharing

DevOps Principles 3 ways
----------------------------------------------
System Thinking (concept to cash) aka overall view
Amplified feedback loop (later you find the bug.. the more it costs to fix)
Work culture to allow for learning and continues experimentation (fail fast, working code wins..sharing..)

Five Methodologies
------------------------------------
People over process over tools
Continues Delivery
Lean management (small batches, progress limits, feedback loops, visualization)
Visible Ops Change Control (repeatable build process, manage dependencies, eliminate fragile artifacts, continues improvement)
Infrastrucure as code

10 practices for DevOps success
------------------------
Incident command system
Developers on call
Status pages (transparent uptime)
Blameless postmortems
Embedded teams
The Cloud
Andon Cords (anyone can stop the release, this way something doesn't go to prod)
Dependency Injection (Inversion of Control)
Blue/Green deployment (2 system, 1 live, the other is not..deploy changes and shift traffic towards it, if problems shift back to prev version)
Chaos Monkey (trash servers occasional so that you can code for it and be prepared in case it happens out of your control.. came from Netflix)

DevOps Tools ... cart or horse?
----------------------------
Toolchain... that works together
Should be programmable.. no UI tools
Verifiable
Well behaved (from dev and operations point of view)
Write own tools if you need to....

TWID March 3, 2019: Death by GPS, Python tool vs code, Alexa + Jenkins, Job Red Flags, Game of Thrones Fantasy League

2019-03-03T08:09:00.000-08:00

This Week I Learned

Finished the Pluralsight course Applying SQL Server 2016 Features to Real-world Data Problems by Ana Voicu

Death by GPS refers to the death of people attributable, in part, to following GPS directions or GPS maps. Death by GPS has been noted in several deaths in Death Valley, California; a lost hiker at Joshua Tree National Park in southeastern California; and incidents in Washington State, Australia, England, Italy and Brazil.

Got to the wikipedia entry after reading some comments on this link on hackernews: The Hunt for the Death Valley Germans

This Week I Tweeted

Python in Visual Studio Code – February 2019 Release

Test Explorer
This release includes the ability to visualize, navigate and run unit tests through a test explorer, a feature that is commonly requested by our users.

Validated breakpoints when debugging
This release also includes an enhancement to the debugger: validation of breakpoints targets. If a breakpoint is set on a line where breakpoints are invalid (e.g. blank lines, pass statement, lines in the middle of a multi-line statement), then it’s automatically moved to the nearest preceeding valid line

The team keeps cranking out more awesomeness every month

Jenkins + Alexa: Say Hello to Voice Controlled CI/CD

Alexa (named after the ancient library of Alexandria) is Amazon’s Artificial Intelligence (AI) powered intelligent voice assistant that runs in the cloud. Software engineers make Alexa smarter by creating apps, called skills. From the time that I developed my first Alexa skill, I dreamed of deploying my Java projects to the cloud via voice. For me, telling Alexa to deploy my code is the ultimate level of cool! I recently made my dream a reality when I devoted a weekend to developing my newest Alexa skill, DevOps Pal. In this blog, I will show you how I developed DevOps Pal and hopefully inspire you to build your own version.

LOL, I have Alexa at home.. but I mostly use it to listen to Music. I am just wondering if they did this because they could and not because this is actually useful. Our builds build upon checkin (on dev, qa/test is scheduled), not sure why you would want to initiate it

Red Flags in Software Developer Job Descriptions

For some reason I’ve been reading a lot of job descriptions for junior devs lately and that has naturally left me with an inordinate number of opinions. I have, with great effort, condensed them here for you, dear reader.

The following aren’t pulled from any specific listings because these types of awfulness transcend the individual - and because that would be dickish. But their spirit should ring true to anyone in the trenches of the job search process.

“Technologies: Coldfusion + jQuery”
Your job search shouldn’t be driven by what’s fashionable in tech, but popularity and the market for a skill are important factors, since they influence a career track’s employability and profitability. The market for Coldfusion devs isn’t too hot (or even a thing) and jQuery simply doesn’t have the complexity or potential power of a full JS MVC. Companies advertising outdated stacks open you up to the risk of building the wrong sorts of skills, which can have far-reaching effects on your career.

One thing that caught my eye was Coldfusion + jQuery, this is the stack that DJindexes website was built on so had to go and troll my ex co-workers haha. One reply I got was: All of these jobs sound very interesting to me. Do they have any COBOL and JCL openings?

See also the comments on hackernews here: https://news.ycombinator.com/item?id=19268989

Motorola confirms its foldable phone is coming

By now, it's not much of a secret that Motorola is working on a folding phone of its own: A patent for such a device surfaced late last year, and a subsequent report in The Wall Street Journal basically confirmed the company's plans. Despite the ensuing hype, Motorola has mostly kept quiet about its progress, but in an interview with Engadget this week, Motorola VP of Global Product Dan Dery shed some additional light on the company's ambitions.

"We started to work on foldables a long time ago," Dery said. "And we have been doing a lot of iteration."

Oh man.. now everyone is jumping on this bandwagon?? What happens when you only crack one screen? This still seems idiotic to me, especially at that price point that Samsung has.

SQL Server 2019 community technology preview 2.3 is now available

This CTP 2.3 preview brings the following new features and capabilities to SQL Server 2019:

Big data clusters

Submit Spark jobs on SQL Server big data clusters from IntelliJ

Application deployment and management experience for deploying a variety of data-related apps including operationalizing machine learning models using R and Python, running SQL Server Integration Services (SSIS) jobs, and more

Database engine

Accelerated database recovery to provide constant time recovery and instant rollback for long running transactions

Performance enhancements in query plan recompilations, transaction log management, and Query Store

SQL Graph enhancement to enable cascaded deletion of edges upon the deletion of nodes

SQL Server Analysis Services (SSAS)

Calculation groups in tabular models that reduce the number of measures by reusing calculation logic

Installed on Sunday morning..checking out what is new and shiny. One thing that Brent Ozar noticed was the is_result_set_caching_on as a new column in sys.databases. So far I haven't heard yet how to turn it on

Some cool stuff you might enjoy

Finished The Annotated Turing: A Guided Tour Through Alan Turing's Historic Paper on Computability and the Turing Machine by Charles Petzold. a little dry at times.. but if you like math or want to learn about Turing machines, this book is for you

There is a Game Of Thrones fantasy league?

I signed up and created a league with a bunch of co-workers
Here is how some of the scoring works

Violence:

10 Kill Random Character / White Walker / Wight (capped at 50pts)
15 Kill White Walker
25 Kill Known or Drafted Character
150 Kill Dragon
5 Incapacitate Random Character (not killed)
10 Incapacitate Known or Drafted Character (not killed)
25 Exit Bonus for Drafted Character Dying Memorably

Status/Power:

200 Take the Iron Throne
25 Sack a City / Lead a Battle Victory (must be present at victory)
25 Get a Seat on the High Council
50 Magic Use (1x Episode / Character)
50 Gain ownership of Valyrian Steel (Must keep through end of episode)
20 Have a Vision/Prophecy (1x / Episode / Character)
15 Take up a Weird/New Religion
10 Get Engaged
20 Get Married
15 Get Pregnant/Get Someone Pregnant
-20 Lose a Baby
25 Official Promotion
-25 Official Demotion
50 Come Back from the Dead
15 Act of Betrayal
15 Form / Join existing alliance
20 Ride or Control Dragon (1x / Episode)
20 MVP of Episode (Scorekeeper's Choice)
50 Revealed as Azor Ahai

I just love Times Square, here is a panorama I took'

Cool day trip in February if you live in New Jersey.. tree tapping at Howell Living History Farm

2019-02-25T07:42:00.002-08:00

Our family went to Howell Living History Farm for the tree tapping event 6 years or so ago, the kids were little and didn't really remember the event. We decided to repeat the event this year so that they would remember this. From our home in Princeton it's about a 25 minute drive, not too bad

It was a cloudy day, so the pics look a little gloomy but it was not too cold

Here you can see some benches that are next to the main entrance where the shop is located

At the shop, you can see a board with all the activities and the time when the activities start. Ask someone for directions if you need to know where to go

While we were walking towards the area where they had the sap collecting buckets, these sheep came running towards us, my kids freaked out but they were very exciting to see the sheep

A little later we saw this big duck, looked like it weighed at least 10 pounds to me, it was about the size of one of those Canada geese birds

The farm had this stack of neatly piled wood

You could actually get busy and cut the wood with this huge saw

These orange roots looked really weird and much better in person

Here you can see the buckets that they use to collect the sap. If you stand still and don't make any noise, you can here the drops making noise when they hit the bucket. It's a pretty interesting sound

A close up shot of the buckets

I tasted the sap as did my wife and the kids. It doesn't taste as syrup at all, it is like water with a very tiny hint of sweetness, the sap is also clear, it does not have a color

This is the evaporator, it is used to make syrup from the sap

Open pan evaporation methods have been streamlined since colonial days, but remain basically unchanged. Sap must first be collected and boiled down to obtain pure syrup without chemical agents or preservatives. Maple syrup is made by boiling between 20 and 50 volumes of sap (depending on its concentration) over an open fire until 1 volume of syrup is obtained, usually at a temperature 4.1 °C (7.4 °F) over the boiling point of water

Boiling the sap for too long will create crystals so you have to be on top of the process and check it. While they were doing the explaining, they also mentioned that the indigenous people would warm up stones and then drop those hot stones in the sap to create syrup.

After you are done with the presentation, you can go to the house where they will make you some old fashioned pancakes.

Farm house

Chickens

These chickens come inside at night because foxes and hawks will snatch them and eat them. They told us to come back in about 4 weeks because that is when they will have the little chicks.

My daughter Catherine petting this horse

After the farm, we drove to Nomad Pizza in Hopewell. If you want to eat there you will probably have to wait since they only have 10 tables or so. In the summer there is an outside area as well. This is why if we go there during the colder months we make sure to get there by 5

I had the chorizo with onion, pepper and mozzarella pizza, it was delicious

TWID Feb 25, 2019: Galaxy Fold, Huawei Mate X, Hololens 2, Juventus, Linux Fsync Issue fix for PostgreSQL, syrup

2019-02-25T06:13:00.001-08:00

This Week I Learned

Almost finished with the book The Annotated Turing: A Guided Tour Through Alan Turing's Historic Paper on Computability and the Turing Machine by Charles Petzold

It takes a lot of maple sap to create maple syrup. The higher the sugar content of the sap, the smaller the volume of sap is needed to obtain the same amount of syrup. 57 units of sap with 1.5 percent sugar content will yield 1 unit of syrup, but only 25 units of sap with a 3.5 percent sugar content are needed to obtain one unit of syrup

Those are my youngest two trying the sap. I tried it as well, I must say there really is no taste to it.

I will make a separate post about our visit to the farm

This Week I Tweeted

Samsung’s foldable phone is the Galaxy Fold, available April 26th starting at $1,980

Samsung first teased its foldable phone back in November, and at the company’s Galaxy Unpacked event today, it’s further detailing its foldable plans. Samsung’s foldable now has a name, the Samsung Galaxy Fold, and the company is revealing more about what this unique smartphone can do. Samsung is planning to launch the Galaxy Fold on April 26th, starting at $1,980, through AT&T and T-Mobile in the US, with a free pair of Samsung’s new wireless earbuds. There will be both an LTE and 5G version of the Galaxy Fold, and Samsung is even planning on launching the device in Europe on May 3rd, starting at 2,000 euros.

Overpriced and if you damage a screen how much to repair this. I rather have a phone like the showed in the Expanse or even better a phone you can roll up so it is the size and shape of a pen. This thing is just too bulky as well. A better design would have been if there was a screen that you could slide out instead.

If you thought the Galaxy Fold was not expensive enough.. no worried The Mate X is Huawei’s 5G foldable phone... the price $2600. Fitting name Mate X, as in mate you will need eXtra money for this one...

Pass on both from me

Juventus share price is down 9% after their champions league result

Ouch, that is not good, but Atletico played a much better 2nd half and converted their chances. Let's see if Juve can advance by scoring at least 2 at home in the return game.

Falsehoods Programmers Believe About Phone Numbers

Some interesting things you might already know, still good to revisit this list

Some cool stuff you might enjoy

Microsoft’s HoloLens 2: a $3,500 mixed-reality headset for the factory

The Microsoft HoloLens 2 is available for preorder today for $3,500, and it’s expected to ship later this year. However, Microsoft has decided that it is only going to sell to enterprise customers who want to deploy the headset to their workers. As of right now, Microsoft isn’t even announcing a developer kit version of the HoloLens 2.

Compared to the HoloLens we first saw demonstrated four years ago, the second version is better in nearly every important way. It’s more comfortable, it has a much larger field of view, and it’s better able to detect real physical objects in the room. It features new components like the Azure Kinect sensor, an ARM processor, eye-tracking sensors, and an entirely different display system.

It has a couple of speakers, the visor flips up, and it can see what your hands are doing more accurately than before. There’s an 8-megapixel front-facing camera for video conferencing, it’s capable of full 6 degrees of tracking, and it also uses USB-C to charge. It is, in short, chock-full of new technology. But after four years, that should be no surprise.

Linux Fsync Issue for Buffered IO and Its Preliminary Fix for PostgreSQL

One of the common fixes applied to all the supported PostgreSQL versions is on – panic instead of retrying after fsync () failure. This fsync failure has been in discussion for a year or two now, so let’s take a look at the implications.

A fix to the Linux fsync issue for PostgreSQL Buffered IO in all supported versions
PostgreSQL performs two types of IO. Direct IO – though almost never – and the much more commonly performed Buffered IO.

PostgreSQL uses O_DIRECT when it is writing to WALs (Write-Ahead Logs aka Transaction Logs) only when wal_sync_method is set to : open_datasync or to open_sync with no archiving or streaming enabled. The default wal_sync_method may be fdatasync that does not use O_DIRECT. This means, almost all the time in your production database server, you’ll see PostgreSQL using O_SYNC / O_DSYNC while writing to WAL’s. Whereas, writing the modified/dirty buffers to datafiles from shared buffers is always through Buffered IO

Starting from kernel 4.13, we can now reliably detect such errors during fsync. So, any open file descriptor to a file includes a pointer to the address_space structure, and a new 32-bit value (errseq_t) has been added that is visible to all the processes accessing that file. With the new minor version for all supported PostgreSQL versions, a PANIC is triggered upon such error. This performs a database crash and initiates recovery from the last CHECKPOINT. There is a patch expected to be released in PostgreSQL 12 that works for newer kernel versions and modifies the way PostgreSQL handles the file descriptors. A long term solution to this issue may be Direct IO, but you might see a different approach to this in PG 12.

Some more info that I found in this hackernews comment thread that might interest you:
If you want an overview of the issue, here's a presentation from Tomas Vondra at FOSDEM 2019: https://youtu.be/1VWIGBQLtxo
Or an early recap of the "fsyncgate" issue in textual form: https://lwn.net/Articles/752063/

Related (also listed by Tomas Vondra): Linux's IO errors reporting https://youtu.be/74c19hwY2oE

As always, I will leave you with a pic I took this past week. This one is a pic of some orange roots over a stream

Calculating Sexy Primes, Prime Triplets and Sexy Prime Triplets in PostgreSQL

2019-02-18T13:56:00.001-08:00

The other day I was reading something on Hackernews and someone posted a link to a Sexy Primes wikipedia article. I looked at that and then decided to do this in SQL Server because.. why not? Then I decided to see how different this would be to do in PostgreSQL. For the first method to create the prime numbers it's different. For the method with the CTE it is very similar

From the Sexy Primes wikipedia link: https://en.wikipedia.org/wiki/Sexy_prime

In mathematics, sexy primes are prime numbers that differ from each other by six. For example, the numbers 5 and 11 are both sexy primes, because 11 minus 5 is 6.

The term "sexy prime" is a pun stemming from the Latin word for six: sex.

If p + 2 or p + 4 (where p is the lower prime) is also prime, then the sexy prime is part of a prime triplet.

Ok I did a couple of versions of this and over the weekend.. here is what I ended up with

So first we need a table that will just have the prime numbers

I decided to populate a table with numbers from 2 till 500 and then use the sieve of Eratosthenes method to delete the non primes

This will look like this

Create this table

CREATE  TABLE  PrimeNumbers  (N INT);

In one window run this to create the function/proc

CREATE OR REPLACE FUNCTION MakePrime() RETURNS void AS $$
DECLARE I integer := 2;
BEGIN
WHILE I <= SQRT(500) LOOP
    DELETE FROM PrimeNumbers WHERE N % I = 0 AND N > I;
    I := I + 1 ; 
END LOOP;

END;

$$ LANGUAGE plpgsql;



In a another window populate the table by making the call to the function
 

INSERT  INTO PrimeNumbers(n)
SELECT N
 FROM (SELECT generate_series(2,500) as n) x;


SELECT MakePrime() ; -- Yes that is a proc call

SELECT * FROM PrimeNumbers

Thinking about it a little more I decided to do it with a CTE instead of a loop with delete statements, if your tables will be big then the delete method is probably better... it's for you to test that out :-)

What we are doing is a NOT EXISTS query against the same cte and we are filtering out numbers that are greater than the number in the current row and are not divisible by the current number

CREATE TABLE IF NOT EXISTS  PrimeNumbers  (N INT);


;WITH cte AS (
  SELECT * FROM generate_series( 2, 500 )  n
  )

INSERT INTO PrimeNumbers
SELECT n
FROM cte
WHERE NOT EXISTS (
  SELECT n FROM  cte as cte2
WHERE cte.n > cte2.n AND cte.n % cte2.n = 0)
;

SELECT * FROM PrimeNumbers;

If we run that last select statement, we should have 95 rows

2
3
5
7
.....
.....
463
467
479
487
491
499

Now that we have our table filled with prime numbers till 500, it's time to run the queries

Sexy prime pairs
The sexy primes (sequences OEIS: A023201 and OEIS: A046117 in OEIS) below 500 are:

(5,11), (7,13), (11,17), (13,19), (17,23), (23,29), (31,37), (37,43), (41,47), (47,53), (53,59), (61,67), (67,73), (73,79), (83,89), (97,103), (101,107), (103,109), (107,113), (131,137), (151,157), (157,163), (167,173), (173,179), (191,197), (193,199), (223,229), (227,233), (233,239), (251,257), (257,263), (263,269), (271,277), (277,283), (307,313), (311,317), (331,337), (347,353), (353,359), (367,373), (373,379), (383,389), (433,439), (443,449), (457,463), (461,467).

Here is that query for the sexy prime pairs

-- 46 rows.. sexy primes
SELECT t1.N,t2.N 
 FROM PrimeNumbers t1
join PrimeNumbers t2 on t2.N - t1.N = 6 
order by 1

It's very simple.. a self join that returns rows where the number from one table alias and the number from the other table alias differ by 6

Prime triplets
The first prime triplets below 500 (sequence A098420 in the OEIS) are

(5, 7, 11), (7, 11, 13), (11, 13, 17), (13, 17, 19), (17, 19, 23), (37, 41, 43), (41, 43, 47), (67, 71, 73), (97, 101, 103), (101, 103, 107), (103, 107, 109), (107, 109, 113), (191, 193, 197), (193, 197, 199), (223, 227, 229), (227, 229, 233), (277, 281, 283), (307, 311, 313), (311, 313, 317), (347, 349, 353), (457, 461, 463), (461, 463, 467)

A prime triplet contains a pair of twin primes (p and p + 2, or p + 4 and p + 6), a pair of cousin primes (p and p + 4, or p + 2 and p + 6), and a pair of sexy primes (p and p + 6).

So we need to check that the 1st and 3rd number have a difference of 6, we also check that that difference between number 1 and 2 is 2 or 4. That query looks like this

-- 22 rows.. Prime Triplets
SELECT t1.N AS N1,t2.N AS N2, t3.N AS N3
 FROM PrimeNumbers t1
join PrimeNumbers t2 on t2.N > t1.N 
join PrimeNumbers t3 on t3.N - t1.N = 6
and t3.N > t2.N
and t2.n - t1.n IN (2,4)
order by 1

Here is what it looks like from pgAdmin

Sexy prime triplets
Triplets of primes (p, p + 6, p + 12) such that p + 18 is composite are called sexy prime. p p, p+6 and p+12 are all prime, but p+18 is not

Those below 500 (sequence OEIS: A046118) are:

(7,13,19), (17,23,29), (31,37,43), (47,53,59), (67,73,79), (97,103,109), (101,107,113), (151,157,163), (167,173,179), (227,233,239), (257,263,269), (271,277,283), (347,353,359), (367,373,379)

The query looks like this.. instead of a self join, we do a triple self join, we also check that p + 18 is not a prime number in the line before the order by

-- 14 rows.. Sexy prime triplets
SELECT t1.N AS N1,t2.N AS N2, t3.N AS N3
 FROM PrimeNumbers t1
join PrimeNumbers t2 on t2.n - t1.n = 6
join PrimeNumbers t3 on t3.N - t1.N = 12
and t3.N > t2.N
AND NOT EXISTS( SELECT null FROM PrimeNumbers p WHERE p.n = t1.n +18)
order by 1

And that's it for this post. If you are interested in the SQl Server version, you can find it here: Calculating Sexy Primes, Prime Triplets and Sexy Prime Triplets in SQL Server

More PostgreSQL posts can be found here: /label/PostgreSQL

TWID Feb 18, 2019: Bruno Ganz, Red hat dropping MongoDB, 500px hacked, VFEmail

2019-02-18T11:14:00.000-08:00

This Week I Learned

Continued reading the book The Annotated Turing: A Guided Tour Through Alan Turing's Historic Paper on Computability and the Turing Machine by Charles Petzold

I was translating a block of T-SQL code that calculated sexy primes into PostgreSQL and found out that in order to use variables, you need to wrap it into a function in PostgreSQL . I also found out that PostgreSQL up until version 11 didn't really have stored procedures either but you could have functions behave like procs

This Week I Tweeted

Hackers wipe US servers of email provider VFEmail

"At this time, the attacker has formatted all the disks on every server," the company said yesterday. "Every VM is lost. Every file server is lost, every backup server is lost."

"This was more than a multi-password via SSH exploit, and there was no ransom. Just attack and destroy," VFEmail said.

Yep someone got pissed of for something and this was a big F U operation

500px Hacked: Personal Data Exposed for All 14.8 Million Users

The popular photo-sharing service 500px has announced that it was the victim of a hack back in 2018 and that personal data was exposed for all the roughly 14.8 million accounts that existed at the time.

In an email sent out to users and an announcement posted to its website, 500px states that it was only on February 8th, 2019, that its team learned of an unauthorized intrusion to its system that occurred on or around July 5th, 2018.

The personal data that may have been stolen by the intruder includes first and last names, usernames, email addresses, password hashes (i.e. not plaintext passwords), location (i.e. city, state, country), birth date, and gender.

Took over 6 months to find out... that is a very long time.. as always make sure that your password is unique for each site that you use

Google will spend $13 billion on U.S. real estate in 2019, expanding into Nevada, Ohio and Texas

CEO Sundar Pichai said in a blog post on Wednesday that the company is building new data centers and offices and expanding several key locations across the U.S., spending $13 billion this year.

Pichai outlined the plans, which include opening new data centers in Nevada, Ohio, Texas and Nebraska, the first time the company will have infrastructure locations in those states. The company is also doubling its workforce in Virginia, providing greater access to Washington, D.C., with a new office and more data center space, and expanding its New York campus at Hudson Square.

Have to spend all that that money to catch up to Amazon and Microsoft in the cloud

Red Hat Satellite to standardize on PostgreSQL backend, will be dropping MongoDB

When will MongoDB Community Edition be dropped as an embedded database within Red Hat Satellite?
This database change is a still to come, but the product team wanted to go ahead and communicate this intent to our users so they were not caught by surprise as this is a change to the underlying databases of Satellite. No specific timing or release is being communicated at this time. At this point we’re simply hoping to raise awareness of the change that is coming to help users of Satellite prepare for the removal of MongoDB.

This is in response to the license changes that MongoDB made recently... Looking at the chart below..it looks like this is no worry to investors, MongoDB just hit a all time high

RIP Bruno Ganz, who Gen X remembers as the angel in "Wings of Desire" and millennials remember as Hitler in that bunker scene.

You have seen all the parodies of course, I actually only watched this movie on January 6th 2018. Downfall is an excellent movie, if you have some time, make sure to watch it

Some cool stuff you might enjoy

Using SonarQube, SonarQube Scanner and the sonar-tsql-plugin to run static code analysis

I wrote this post for a friend so that he has a reference on how to install SonarQUbe and how to get started. This post explain how you can user SonarQube to run static code analysis against your T-SQL procs and functions

February release of @AzureDataStudio is now available!

- Admin Pack for SQL Server extension
- Auto-sizing columns in results
- Notebook UI improvements
- Profiler Filtering
- Save Results as XML
- Deploy scripts

I am still using SSMS but maybe I will switch to DataStudio one of these days

Someone took 50,000 images of the night sky to make an 81 Megapixel image of the moon It's beautiful
See it here: https://www.reddit.com/r/space/comments/arer0k/i_took_nearly_50000_images_of_the_night_sky_to/ …

mirrors of both JPG and PNG in zoomable versions here:

https://micr.io/i/clIZW (JPG)
https://micr.io/i/WFjqr (PNG)

Finding rows where the column starts or ends with a 'bad' character

Another post I wrote because of a problem that a co-worker had with some data

A nice view while going to the Princeton Junction train station... had to take a pic

Twid Feb 10 2019: Gemini, Stablecoin, BrightBytes, Turing, world war I restored,githistory

2019-02-09T12:38:00.000-08:00

This Week I Learned

Continued reading the book The Annotated Turing: A Guided Tour Through Alan Turing's Historic Paper on Computability and the Turing Machine by Charles Petzold

Technically this didn't happen this week, but I decided to blog about it this week: After 20+ years in IT .. I finally discovered this...

This Week I Tweeted

Microsoft buys BrightBytes DataSense to bring more data analytics to schools

BrightBytes, based in San Francisco, is an education data-analytics company and has been a Microsoft partner for years. In addition to the DataSense data-integration platform it is selling to Microsoft, BrightBytes will continue to sell its decision-support platform called Clarity which uses machine learning and predictive analytics.

Microsoft is planning to integrate DataSense into its Microsoft Education product family. According to Microsoft, DataSense is "the leading IPaaS (integration platform as a service) solution for both education solution providers and school districts across the US."

I don't know much about this company.. but it's the data and analytics that is valuable... companies like this one are getting snapped up left and right

Winklevoss Exchange Gemini Shuts Down Accounts Over Stablecoin Redemptions

In one instance, email correspondence obtained by CoinDesk shows an OTC trader based in Latin America had his account closed after he informed Gemini that he planned to redeem several million dollars of GUSD. (A major cryptocurrency exchange, speaking on condition of anonymity, attested to the desk’s professionalism and reported that it was in good standing.)

So you can deposit money but don't you dare take it out........

Python Developers Survey 2018 Results

Python usage as a main language is up 5 percentage points from 79% in 2017 when Python Software Foundation conducted its previous survey.

Half of all Python users also use JavaScript. The 2018 stats are very similar to the 2017 results. The only significant difference is that Bash/Shell has grown from 36% in 2017 to 45% in 2018. Go and SQL have also grown by 2 percentage points each, while many other languages such as C/C++, Java, and C# have lost their share.

In 2018 we had significantly more respondents specifying they’re involved in DevOps (an increase of 8% compared to 2017). In terms of Python users using Python as their secondary language, DevOps has overtaken web development.

The use of Python 3 continues to grow rapidly. According to the latest research in 2017, 75% were using Python 3 compared with 25% for Python 2. Use of Python 2 is declining as it’s no longer actively developed, doesn’t get new features, and its maintenance is going to be stopped in 2020.

See all the other interesting fact on the jetbrains site

Githistory
This is pretty cool.. point any public github file here and you can see the last 8 or so commits by scrolling from the top here is what @BrentOzarULTD sp_foreachdb changes look like ...

It looks like this

So for any file, just replace github.com in the url with githistory.xyz

For example.. the stuff in red needs to be replaced

https://github.com/BrentOzarULTD/SQL-Server-First-Responder-Kit/blob/dev/sp_foreachdb.sql
https://githistory.xyz/BrentOzarULTD/SQL-Server-First-Responder-Kit/blob/dev/sp_foreachdb.sql

Just hit this link to see it

https://githistory.xyz/BrentOzarULTD/SQL-Server-First-Responder-Kit/blob/dev/sp_foreachdb.sql

Some cool stuff you might enjoy

List of stories set in a future now past

This is a dynamic list and may never be able to satisfy particular standards for completeness. You can help by expanding it with reliably sourced entries.
This is a list of fictional stories that, when written, were set in the future, but the future they predicted is now present or past. The list excludes works that were alternate histories, which were composed after the dates they depict. The list also excludes contemporary or near-future works (e.g. set within a year or two), unless it deals with some notable futuristic event as with the 2012 phenomenon. It also excludes works where the future is passively mentioned and not really depicting anything notable about the society, as with an epilogue that just focuses on the fate of the main characters. Entries referencing the current year may be added if their month and day were not specified or have already occurred.

Some from the year 2019... Akira, Blade Runner, Dark Angel

How Peter Jackson’s team made World War I footage look new

“I thought, ‘Can we actually make this 100-year-old footage look like it was shot now?’” Jackson said on the latest episode of Recode Decode with Kara Swisher. “So it’s sharp, it’s clear, it’s stable, it looks like modern [films].”

And he had the means to do that: Over the past five years, Jackson tasked Park Road Post (a subsidiary of his production company WingNut Films) with adding color and sound to the archive footage, as well as making the frame rate consistent and similar to what we’d expect from footage shot today. The result is the new documentary “They Shall Not Grow Old,” which he said removes the “barrier between us and the actual people that were being filmed.”

This is some amazingly looking stuff... here is a video so you can see what it looks like

Build 2019 registration opens on February 27th

Join us in Seattle for Microsoft’s premier event for developers. Come and experience the latest developer tools and technologies. Imagine new ways to create software by getting industry insights into the future of software development. Connect with your community to understand new development trends and innovative ways to code.

Wondering what new stuff they will announce

The Chemical Brothers: Setting Sun (1996)
Haven't heard Setting Sun for a while, it's an excellent running song, I have added it to my running playlist on my phone

Setting Sun is a song by The Chemical Brothers with vocals by Noel Gallagher from the group Oasis. It was released as a single in 1996 from their second album Dig Your Own Hole and reached number one on the UK Singles Chart.

Took this on my way to the Princeton Junction train station... beautiful morning colors

TWID Feb 3rd 2019, Huawei stealing, billion solar panels, big puzzle, beethoven, kodak

2019-02-04T05:52:00.001-08:00

This Week I Learned

Continued with the Getting Started with Docker Swarm Mode Pluralsight course

Started reading the book The Annotated Turing: A Guided Tour Through Alan Turing's Historic Paper on Computability and the Turing Machine by Charles Petzold

This Week I Tweeted

Huawei is accused of attempting to copycat a T-Mobile robot, and the charges read like a comical spy movie

The US on Monday charged the Chinese phone giant Huawei with trying to steal trade secrets from T-Mobile, among other crimes.
One Justice Department indictment includes internal emails between Huawei's US and Chinese employees who prosecutors said were trying to copy a T-Mobile device-testing robot.
The emails read like a comical spy movie, with one set of employees trying to avoid wrongdoing and another engineer getting caught putting part of the robot into his bag.
Huawei said that it hasn't violated any US laws and that it already settled with T-Mobile in a civil lawsuit.

What the heck is going on here?

How Do You Count Every Solar Panel in the U.S.? Machine Learning and a Billion Satellite Images

The DeepSolar Project, developed by engineers and computer scientists at Stanford University, is a machine learning framework that analyzes a dataset of satellite images in order to identify the size and location of installed solar panels.

To accurately count the panels, the DeepSolar team used a machine learning algorithm to analyze more than a billion high-resolution satellite images. The algorithm identified what the team believes to be almost every solar power installation across the contiguous 48 states.

The DeepSolar analysis reached a total of 1.47 million solar installations in the U.S., a much higher number than either of the two most commonly cited estimates.

“We can use recent advances in machine learning to know where all these assets are, which has been a huge question, and generate insights about where the grid is going and how we can help get it to a more beneficial place,” said Ram Rajagopal, associate professor of civil and environmental engineering, who supervised the project with Arun Majumdar, professor of mechanical engineering.

This is cool..but do they really need to use a billion images.. can't they use a subset and come pretty close to the real answer as well?

40x faster hash joiner with vectorized execution

For the past four months, I’ve been working with the incredible SQL Execution team at Cockroach Labs as a backend engineering intern to develop the first prototype of a batched, column-at-a-time execution engine. During this time, I implemented a column-at-a-time hash join operator that outperformed CockroachDB’s existing row-at-a-time hash join by 40x. In this blog post, I’ll be going over the philosophy, challenges, and motivation behind implementing a column-at-a-time SQL operator in general, as well as some specifics about hash join itself.

In CockroachDB, we use the term “vectorized execution” as a short hand for the batched, column-at-a-time data processing that is discussed throughout this post.

I love it when you see drastic speed improvements like this, I remember one time when we upgraded some hardware to use SSDs and more RAM.. a reporting query that took a minute now finished in less than a second.. I thought the results were wrong because it finished too fast lol

Some cool stuff you might enjoy

Kodak Premium Puzzle Presents: The World's Largest Puzzle 51,300 Pieces 27 Wonders from Around The World 28.5 Foot x 6.25 Foot Jigsaw Puzzle

That is 8.68 meters wide for the metric people....aka 1 big white shark length

I don't always listen to classical music.. but when I do.. I make sure it's played on an electric guitar, enjoy this piece of Ludwig van Beethoven - Moonlight Sonata ( 3rd Movement ) by Tina S

TWID Jan 27th, Salah, SSMS emoji, franz josef, mourning stamps,american gods season 2, PostgreSQL

2019-01-27T06:57:00.001-08:00

This Week I Learned

Continued with the Getting Started with Docker Swarm Mode Pluralsight course
Finished A History of Japan by R.H.P. And J. G. Caiger Mason

Did you know, you can use emojis in output from SSMS?

More info, including the code to run can be found here: Did you know, you can use emojis in output from SSMS?

This Week I Tweeted

Why are glasses so expensive? The eyewear industry prefers to keep that blurry

It’s a question I get asked frequently, most recently by a colleague who was shocked to find that his new pair of prescription eyeglasses cost about $800. Why are these things so damn expensive? The answer: Because no one is doing anything to prevent a near-monopolistic, $100-billion industry from shamelessly abusing its market power. Prescription eyewear represents perhaps the single biggest mass-market consumer ripoff to be found. The stats tell the whole story.

Totally agree

The Internals of PostgreSQL for database administrators and system developers

In this document, the internals of PostgreSQL for database administrators and system developers are described.

PostgreSQL is an open source multi-purpose relational database system which is widely used throughout the world. It is one huge system with the integrated subsystems, each of which has a particular complex feature and works with each other cooperatively. Although understanding of the internal mechanism is crucial for both administration and integration using PostgreSQL, its hugeness and complexity prevent it. The main purposes of this document are to explain how each subsystem works, and to provide the whole picture of PostgreSQL.

This document is based on the second part of the book I wrote in Japanese in 2012 (ISBN-13: 978-4774153926) which is composed of seven parts, and covers version 11 and earlier.
Contents

Chapter 1. Database Cluster, Databases and Tables
Chapter 2. Process and Memory Architecture
Chapter 3. Query Processing
Chapter 4. Foreign Data Wrappers (FDW) and Parallel Query
Chapter 5. Concurrency Control
Chapter 6. VACUUM Processing
Chapter 7. Heap Only Tuple (HOT) and Index-Only Scans
Chapter 8. Buffer Manager
Chapter 9. Write Ahead Logging (WAL)
Chapter 10. Base Backup and Point-In-Time Recovery (PITR)
Chapter 11. Streaming Replication

What are you wating for.. start reading.....

Since we are talking about PostgreSQL.....

Microsoft acquires Citus Data, re-affirming its commitment to Open Source and accelerating Azure PostgreSQL performance and scale

I am thrilled to announce that we have acquired Citus Data, a leader in the PostgreSQL community. Citus is an innovative open source extension to PostgreSQL that transforms PostgreSQL into a distributed database, dramatically increasing performance and scale for application developers. Because Citus is an extension to open source PostgreSQL, it gives enterprises the performance advantages of a horizontally scalable database while staying current with all the latest innovations in PostgreSQL. Citus is available as a fully-managed database as a service, as enterprise software, and as a free open source download.

Since the launch of Microsoft’s fully managed community-based database service for PostgreSQL in March 2018, its adoption has surged. Earlier this month, PostgreSQL was named DBMS of the Year by DB-Engines, for the second year in a row. The acquisition of Citus Data builds on Azure’s open source commitment and enables us to provide the massive scalability and performance our customers demand as their workloads grow.

Together, Microsoft and Citus Data will further unlock the power of data, enabling customers to scale complex multi-tenant SaaS applications and accelerate the time to insight with real-time analytics over billions of rows, all with the familiar PostgreSQL tools developers know and love.

That's pretty cool..hopefully they will leave the team alone and not do a FoxPro type move..... But since Ballmer is not in charge anymore, I have faith

Massive mortgage and loan data leak gets worse as original documents also exposed

The bucket contained 21 files containing 23,000 pages of PDF documents stitched together — or about 1.3 gigabytes in size. Diachenko said that portions of the data in the exposed Elasticsearch database on Wednesday matched data found in the Amazon S3 bucket, confirming that some or all of the data is the same as what was previously discovered. Like in Wednesday’s report, the server contained documents from banks and financial institutions across the U.S., including loans and mortgage agreements. We also found documents from the U.S. Department of Housing and Urban Development, as well as W-2 tax forms, loan repayment schedules and other sensitive financial information.

This is really, really messed up

Google Search Operators: The Complete List (42 Advanced Operators)

Did you know that Google is constantly killing useful operators?
That’s why most existing lists of Google search operators are outdated and inaccurate.
For this post, I personally tested EVERY search operator I could find.
Here is a complete list of all working, non‐working, and “hit and miss” Google advanced search operators as of 2018.

All you need to step up your Google Fu

Some cool stuff you might enjoy

Austro-Hungarian emperor Franz Josef defaced

See more info here: The revenge of Yugoslavia on Bosnian stamps of 1906

If you are interested in philately, see also, why do some of these stamps have black perforations?

Took this pic of Liverpool player Mo Salah mural near time square

American Gods | Season 2 Official Trailer

Can't wait to watch that, have watched season I and have also read the book last year

TWID Jan 20th 2019.. ... Crypts of Winterfell, Brexit, re:MARS, remars, myths, trace flags, agile bs

2019-01-20T07:02:00.001-08:00

This Week I Learned

Continued with the Getting Started with Docker Swarm Mode Pluralsight course
Finished D'Aulaires' Book of Greek Myths by Ingri d'Aulaire, Edgar Parin d'Aulaire

This Week I Tweeted

MSSQL Tiger Team: Let’s talk about trace flags

So why do we even have trace flags to begin with?
Some trace flags are used to enable enhanced debugging features such as additional logging, memory dumps etc. and are used only when you are working with Microsoft Support to provide additional data for troubleshooting. These trace flags are not ones you want to leave turned on in a production system as they may have a negative impact on your workload. An example of one of these flags would be TF 2551 which is used to trigger a filtered memory dump whenever there is an exception or assertion in the SQL Server process. These trace flags are only used for a short period of time and typically only at the recommendation of Microsoft Support, so they will likely always be around.

Other trace flags are used to alter the behavior of the server in other ways, such as to turn a feature ON or OFF or change the way the database engine manages resources. One example of this type of trace flag is 7752 which was introduced as a knob for something that is default behavior in Azure SQL DB. In a SQL Server (on-prem or IaaS) database that is undergoing recovery and has Query Store (QDS) enabled, user queries will be blocked until all the data required for QDS to start is loaded. This ensures QDS doesn't miss any queries that are executed in that database. In some cases, this can take a long time to complete and you'll see sessions with a wait type of QDS_LOADDB until the QDS becomes available. Turning on TF 7752 makes this process asynchronous so that user queries can proceed while the QDS starts. It's not something we want to make the default behavior in general because it means that some query executions won't be captured by QDS, but it's a tradeoff you might be willing to make in order to reduce the time it takes for a database to become available upon restart or failover. This is the sort of trace flag that we are trying to incorporate into the product in some other way, such as to provide a database-scoped configuration. Moving forward, we hope not to introduce any new trace flags like this, and over time the number of flags of this type should approach zero.

Make sure to read the rest of the post and read the links to the knowledge base articles as well

Broadcast group Discovery shifts jobs from London to Amsterdam

Discovery has become the latest international broadcast group to move European operations to the Netherlands following Britain’s decision to pull out of the EU. Discovery has had a Dutch base since 1989 but has now applied for EU licences for its paid channel portfolio through the Netherlands.

This decision ensures continuity of our services for the view across Europe,’ the company said in a statement.

I have seen a lot of these job shifts over the last year because of Brexit, even the company I work for has moved people to Amsterdam from London. This will be very costly for London.

Bullet Journal Progress Walls and Progress Grids

The one thing I like to have in my Bullet Journal is a progress wall or progress grid
What a progress wall can do for you is to visually let you know how long you are from your goal. You can make it look neat and pretty or as in my case a little messy

Detecting Agile BS(from the US Department Of Defense) [pdf]

Agile is a buzzword of software development, and so all DoD software development projects
are, almost by default, now declared to be “agile.” The purpose of this document is to provide
guidance to DoD program executives and acquisition professionals on how to detect software
projects that are really using agile development versus those that are simply waterfall or spiral
development in agile clothing (“agile-scrum-fall”).

Pretty cool from the department of Defense

MongoDB removed from RHEL 8 beta due to license

Note that the NoSQL MongoDB database server is not included in RHEL 8.0 Beta because it uses the Server Side Public License (SSPL).

Didn't AWS roll out their own version as well?

Lego collecting delivers huge and uncorrelated market returns

A lot of fancy things can be built with Lego sets nowadays. Such as a diversifying portfolio that loads on the Fama-French size factor. Collecting Lego -- yes, the plastic toys made of interlocking bricks that become cars and castles and robots -- returned more than large stocks, bonds and gold in the three decades ending in 2015, says a study by Victoria Dobrynskaya, an assistant professor at Russia’s Higher School of Economics. Aspects of the performance even align with returns sought by owners of smart-beta ETFs.

Man, I wish I still had my space sets from the 70s intact

Some cool stuff you might enjoy

Apple Open-Sources FoundationDB Record Layer

Today, we are excited to announce the open source release of the FoundationDB Record Layer!

From its inception, FoundationDB was designed as a highly scalable key-value store with a simple API. Layers extend the functionality of the database by adding features and data models to allow new storage access patterns. Today we are releasing the FoundationDB Record Layer, which provides relational database semantics on top of FoundationDB. This layer features schema management, indexing facilities, and a rich set of query capabilities. The Record Layer is used in production at Apple to support applications and services for hundreds of millions of users.

Supports ACID transactions

re:MARS, a new AI event for machine learning, automation, robotics, and space

Machine Learning (ML) and Artificial Intelligence (AI) are behind almost everything we do at Amazon. Some of this work is highly visible, such as autonomous Prime Air delivery drones, eliminating checkout lines at Amazon Go, and making everyday life more convenient for customers with Alexa. But much of what we do with AI and ML happens beneath the surface – from the speed in which we deliver packages, to the broad selection and low prices we’re able to offer customers, to automatic extraction of characters and places from books and videos with X-Ray. This is in addition to the unrivaled breadth and depth of AI and ML services that AWS offers businesses of all sizes. And it’s still day one for AI at Amazon.

Today we are excited to announce we are bringing together some of the brightest leaders across science, academia, and business to explore innovation, scientific advancements, and practical applications of AI and ML. re:MARS, Amazon’s new AI event focused on Machine learning, Automation, Robotics, and Space, will take place June 4-7, 2019 at the ARIA Resort & Casino in Las Vegas. Business leaders and technical builders will learn, share, and further imagine how these four fields of study will shape the future of AI.

See also here: https://remars.amazon.com/

Game of Thrones | Season 8 | Official Tease: Crypts of Winterfell (HBO)

We have all been waiting for this... but where are the books??

Bullet Journal Progress Walls and Progress Grids

2019-01-14T12:10:00.002-08:00

I got a Bullet Journal as a gift in December and have been messing around with it a little
The one thing I like to have in my Bullet Journal is a progress wall or progress grid
What a progress wall can do for you is to visually let you know how long you are from your goal. You can make it look neat and pretty or as in my case a little messy

I decided to run 750 miles this year and also do 15,000 push ups. This means I will do about 100 push ups every other day and run about 4 miles 4 times a week

What I did for the 17,500 push ups is divide it up by 100 so that I need a wall with 175 bricks

For the 750 mile run, I divided it by 2.5 miles so that I ended up with 300 bricks

Once you are done with fill in the brick in your favorite color, I chose the color red

Here is what my messy walls look like

Once you finish a row, put a date next to it, this way you can quickly see when you completed that section

If you prefer a neater version, you can of course do a grid, here is what the progress grid I created for my SQL Server blog posts looks like

My goal is to write 100 SQL Server Posts in 2019, here is what the grid looks like right now, so far I created 4 post, I have colored those brick in red

You can use a progress wall for all kind of things... maybe you are saving for a vacation and putting $20 away each week... divide the amount you need by 20 and you have how many bricks you need in your wall.

A progress wall is a fun and creative way to keep track of how far you are from your goal

Let me know in a comment what you would use a progress wall for to track progress

TWID Jan 13th 2019.. Rules of 3. hacked photosynthesis. Containers Killed The Virtual Machine

2019-01-13T11:26:00.000-08:00

This Week I Learned

Finished the Getting Started with Docker Pluralsight course

Rules of 3... for survival
3 Minutes for air
3 Days for water
3 Weeks for food

This Week I Tweeted

Scientists Have 'Hacked Photosynthesis' In Search Of More Productive Crops

"This is a very important finding," she says. "It's really the first major breakthrough showing that one can indeed engineer photosynthesis and achieve a major increase in crop productivity."

It will be many years, though, before any farmers plant crops with this new version of photosynthesis. Researchers will have to find out whether it means that a food crop like soybeans actually produces more beans — or just more stalks and leaves.

Then they'll need to convince government regulators and consumers that the crops are safe to grow and eat.

Giant CES fair in Vegas shows US remains too big to ignore for Chinese tech firms even as trade war swirls

Times are tough in 2019 thanks to the US-China trade war and an escalating war of words between Washington and Beijing over tech leadership

Chinese companies at CES all agreed though that while the trade war has adversely impacted their business in the US, it remains a very important market

“We are definitely affected by the tariffs, in fact one of our big US customers is moving their manufacturing operations outside of China to Vietnam to avoid an increase in the cost of doing business,” said Yuki, a saleswoman from Dongguan-based Ruiheng Electronic Co. Ltd., which manufactures power adaptors and circuit boards.

Costco Sells Out of 27-Pound ‘Storage Bucket’ Mac and Cheese With 20-Year Shelf Life

Bad news for those looking to stock up on Costco’s food hit, the 27-pound bucket of macaroni and cheese with 20 years of shelf life: it’s now sold out on the warehouse’s website after going viral on social media.

Who buys this stuff?

Containers Killed The Virtual Machine Star

We predict new enterprise application development will pass a tipping point in 2019 and shift away from legacy virtual machines (VMs) and strongly toward containers and Kubernetes container orchestration.

To be precise, we predict that:

The future is multi-cloud, and multi-cloud means Docker containers with Kubernetes orchestration. Every public cloud has its own APIs, and in that sense, they are all new versions of proprietary mainframes.
“Lift-and-shift” of VMware virtual machines will be more expensive than customers realize. Paying no money up front, in this case to refactor and port applications to Kubernetes, typically means paying more during operations.
Java is not dead. It may play an important middle ground between lift-and-shift and the expense of completely refactoring applications for cloud-native environments. Java may be a light touch version of “move-and-improve.”
It seems likely that Lenovo will take a look at acquiring SUSE, with Supermicro perhaps also in the mix.

Docker/Kubernetes and other container technologies is all the rage now...especially with DevOps

Print.. the disruptor of batch deletes in SQL

A SQL Server post showing you how the print statement might mess up your batch delete statement

Dozens of government websites rendered insecure or inaccessible due to 80+ expired TLS certificates that haven't been renewed due to the US government shutdown

As the U.S. federal shutdown continues, dozens of U.S. government websites have been rendered either insecure or inaccessible due to expired transport layer security (TLS) certificates that have not been renewed.

In fact, .gov websites are using more than 80 TLS certificates that have expired, according to a new Thursday report by Netcraft. That’s because funding for renewals has been paused. That opens the impacted sites to an array of cyber-attacks; most notably, man-in the-middle attacks, which allow bad actors to intercept exchanges between a user and a web application—either to eavesdrop or to impersonate the website and steal any data that the user may input.

That's not good... shouldn't these people be on the essential employee list?

Some cool stuff you might enjoy

500 Top PDFs posted to Hacker News in 2018

The top 5

1. Norwegian Consumer Council report on how tech companies use dark patterns [pdf]
score: 661 comments
2. Assembly Language for Beginners [pdf]
score: 590 comments
3. The Periodic Table of Data Structures [pdf]
score: 534 comments
4. Competitive Programmer's Handbook (2017) [pdf]
score: 514 comments
5. DEF CON report on vulnerabilities in US election infrastructure [pdf]

Rest can be found here: https://getpolarized.io/2019/01/08/top-pdfs-of-2018-hackernews.html

Mongo song....

Hey Mongo I just met you And this is craaazy but here's my data so store it maybe?

B bu but.. it's webscale :-)

The ideal amount of sunlight for growing your garden

See the infographic here: https://www.dailyinfographic.com/wp-content/uploads/2018/10/Garden-sunlight-infographic.jpg

Roadmap to becoming a Web Developer in 2019

Seems very overwhelming to me

See it all here: https://github.com/kamranahmedse/developer-roadmap

.

Alexander Hall as seem from Blair Hall at Princeton University

Took this on my way to pick up the kids from an event

Using PostgreSQL's Interval to mimic SQL Server's DATEADD function

2018-12-03T08:26:00.003-08:00

I wanted to do some date calculations in PostgreSQL and was doing some research on if something like the DATEDIFF function that exists in SQL Server is available in PostgreSQL. These notes are mostly for me so I can refer back to them..but maybe they are useful for someone else as well

In SQL Server to add dates, minutes and other fractions of a date to a date, you can use the DATEADD function

Here are some quick examples, if you want to run this in SQL Server, create this table with one row first

CREATE  TABLE test(SomeDate date);

INSERT INTO test
VALUES('20120101');

And here is a simple DATEADD query, that adds 1 or 2 days to a date by using both the datepart and the abbreviated datepart

SELECT SomeDate,
  DATEADD(dd,1,SomeDate) as Interval1dd,
  DATEADD(dd,2,SomeDate) as Interval2dd,
  DATEADD(day,1,SomeDate) as Interval1Day,
  DATEADD(day,2,SomeDate) as Interval2Day
FROM test

That will give us the following output

SomeDate Interval1dd Interval2dd Interval1Day Interval2Day

2012-01-01 2012-01-02 2012-01-03 2012-01-02 2012-01-03

If you want to go negative, all you have to do is place a minus sign in front of the number

SELECT SomeDate,
  DATEADD(dd,-1,SomeDate) as Interval1dd,
  DATEADD(dd,-2,SomeDate) as Interval2dd,
  DATEADD(day,-1,SomeDate) as Interval1Day,
  DATEADD(day,-2,SomeDate) as Interval2Day
FROM test

Here is the output of that query

SomeDate Interval1dd Interval2dd Interval1Day Interval2Day
2012-01-01 2011-12-31 2011-12-30 2011-12-31 2011-12-30

Here are all the valid datepart arguments in SQL Server

datepart	Abbreviations
year	yy, yyyy
quarter	qq, q
month	mm, m
dayofyear	dy, y
day	dd, d
week	wk, ww
weekday	dw, w
hour	hh
minute	mi, n
second	ss, s
millisecond	ms
microsecond	mcs
nanosecond	ns

Now let's take a look at PostgreSQL

In PostgreSQL, there is no DATEPART function, but you can use interval literals to accomplish something that behaves the same

Let's do something similar like we did with the SQL Server queries, first create this temp table with one row

CREATE Temp TABLE test(SomeDate date);


INSERT INTO test
VALUES( to_date('20120101','YYYYMMDD'));

Now let's run this query

SELECT SomeDate,(SomeDate + 1 * INTERVAL '1 Day' ) as Interval1Day,
  (SomeDate + 1 * INTERVAL '1 D')  as IntervalD,
  (SomeDate + 2 * INTERVAL '1 Day' ) as Interval2Times1Day,
  (SomeDate + 1 * INTERVAL '2 Day' ) as Interval2Days,
  (SomeDate + 2 * INTERVAL '2 Days' ) as Interval2Times2Days,
  (SomeDate +  INTERVAL '1 Day' )    as IntervalD
FROM test

Here is the output

"2012-01-01";"2012-01-02 00:00:00";"2012-01-02 00:00:00";"2012-01-03 00:00:00";"2012-01-03 00:00:00";"2012-01-05 00:00:00";"2012-01-02 00:00:00"

When you copy from pgAdmin , you don't get the column aliases, so below is a screenshot of what it looks like(Click on the image for a bigger sized image)

As you can see there are two parts where you can supply a number

I think I prefer the top one from the query below, since it resembles the DATEPART function more

(SomeDate + 2 * INTERVAL '1 Day' ) as Interval2Times1Day,
(SomeDate + 1 * INTERVAL '2 Day' ) as Interval2Days,

But as you saw,it was possible to add 4 days by using a one in both places like shown below

(SomeDate + 2 * INTERVAL '2 Days' ) as Interval2Times2Days,

And of course if you want, you can just use the number inside the string like in this example below

(SomeDate +  INTERVAL '1 Day' )    as IntervalD

It's up to you, but I don't like changing numbers inside a string

To do negative numbers, you just change the positive number to a negative number, here is the same query from before but now with negative numbers

SELECT SomeDate,(SomeDate + -1 * INTERVAL '1 Day' ) as Interval1Day,
  (SomeDate + -1 * INTERVAL '1 D')    as IntervalD,
  (SomeDate + -2 * INTERVAL '1 Day' ) as Interval2Times1Day,
  (SomeDate + -1 * INTERVAL '2 Day' ) as Interval2Days,
  (SomeDate + -2 * INTERVAL '-2 Days')as Interval2Times2Days,
  (SomeDate +  INTERVAL '-1 Day' )    as IntervalD
FROM test

Here is the output
"2012-01-01";"2011-12-31 00:00:00";"2011-12-31 00:00:00";"2011-12-30 00:00:00";"2011-12-30 00:00:00";"2012-01-05 00:00:00";"2011-12-31 00:00:00"

As you can see that all works as expected, did you notice that the we get +4 when we do -2 * -2?

Here is the output also from pgAdmin so that you can see the column aliases

Besides using days, you can also use these parts of a date

Abbreviation	Meaning
Y	Years
M	Months (in the date part)
W	Weeks
D	Days
H	Hours
M	Minutes (in the time part)
S	Seconds

Let's take a look by using some of these in a query

SELECT SomeDate,(SomeDate + 1 * INTERVAL '1 Day' ) as Interval1Day,
  (SomeDate + 1 * INTERVAL '1 Week' ) as Interval1Week,
  (SomeDate + 1 * INTERVAL '1 Month' ) as Interval1Month,
  (SomeDate + 1 * INTERVAL '1 Year' ) as Interval1Year
  
FROM test
UNION ALL
SELECT SomeDate,(SomeDate + 1 * INTERVAL '1 D' ) as Interval1Day,
  (SomeDate + 1 * INTERVAL '1 W' ) as Interval1Week,
  (SomeDate + 1 * INTERVAL '1 M' ) as Interval1Month,
  (SomeDate + 1 * INTERVAL '1 Y' ) as Interval1Year
  
FROM test

Here is the output

"2012-01-01";"2012-01-02 00:00:00";"2012-01-08 00:00:00";"2012-02-01 00:00:00";"2013-01-01 00:00:00"
"2012-01-01";"2012-01-02 00:00:00";"2012-01-08 00:00:00";"2012-01-01 00:01:00";"2013-01-01 00:00:00"

Here is the output also from pgAdmin so that you can see the column aliases

As you can see when using M it used minute not month. I would recommend to always use the full name and not the abbreviated part so as not to create confusion

That's all for this post

Easy running totals with windowing functions in PostgreSQL

2018-11-25T12:41:00.000-08:00

Back in the pre windowing function days, if you wanted to do a running count, you either had to run a subquery or you could use a variable. This was slow because for each row the query that did the sum would be executed. With windowing functions in PostgreSQL, this is now running much faster.

Let's take a look, first create the following table

CREATE Temp TABLE test(Id int,SomeDate date, Charge decimal(20,10));


insert into test
values( 1,to_date('20120101','YYYYMMDD'),1000);
insert into test
values( 1,to_date('20120401','YYYYMMDD'),200);
insert into test
values( 1,to_date('20120501','YYYYMMDD'),300);
insert into test
values( 1,to_date('20120601','YYYYMMDD'),600);
insert into test
values( 2,to_date('20120101','YYYYMMDD'),100);
insert into test
values( 2,to_date('20120101','YYYYMMDD'),500);
insert into test
values( 2,to_date('20120101','YYYYMMDD'),-800);
insert into test
values( 3,to_date('20120101','YYYYMMDD'),100);

let's check that data we just inserted into the temporary table

SELECT * FROM test

The output looks like this

Id SomeDate Charge
1 2012-01-01 1000.0000000000
1 2012-04-01 200.0000000000
1 2012-05-01 300.0000000000
1 2012-06-01 600.0000000000
2 2012-01-01 100.0000000000
2 2013-01-01 500.0000000000
2 2014-01-01 -800.0000000000
3 2012-01-01 100.0000000000

What we want is the following

id StartDate Enddate         Charge         RunningTotal
1 2012-01-01 2012-03-31 1000.0000000000 1000.0000000000
1 2012-04-01 2012-04-30 200.0000000000 1200.0000000000
1 2012-05-01 2012-05-31 300.0000000000 1500.0000000000
1 2012-06-01 9999-12-31 600.0000000000 2100.0000000000
2 2012-01-01 2012-12-31 100.0000000000 100.0000000000
2 2013-01-01 2013-12-31 500.0000000000 600.0000000000
2 2014-01-01 9999-12-31 -800.0000000000 -200.0000000000
3 2012-01-01 9999-12-31 100.0000000000 100.0000000000

For each row, we want to have the date that the row starts on and also the date when it ends, we also want a running total as well. If there is no row after the current row for that id, we want the end date to be 9999-12-31.

So we will use a couple of functions. The first one is LEAD, LEAD accesses data from a subsequent row in the same result set without the use of a self-join. So the LEAD part looks like this

LEAD((SomeDate + -1 * INTERVAL '1 day' ),1,'99991231') OVER (PARTITION BY id ORDER BY SomeDate) as Enddate,

What we are doing is subtracting 1 from the date in the subsequent row (SomeDate + -1 * INTERVAL '1 day' )
We are using 1 as the offset since we want to apply this to the next row. Finally if there is no subsequent row, we want to use the date 9999-12-31 instead of NULL

To do the running count, we will do the following

SUM(Charge) OVER (PARTITION BY id ORDER BY SomeDate
ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW)
AS RunningTotal

What this means in English is for each id ordered by date, sum up the charge values for the rows between the preceding rows and the current row. Here is what all that stuff means.

ROWS BETWEEN
Specifies the rows that make up the range to use as implied by

UNBOUNDED PRECEDING
Specifies that the window starts at the first row of the partition. UNBOUNDED PRECEDING can only be specified as window starting point.

CURRENT ROW
Specifies that the window starts or ends at the current row when used with ROWS or the current value when used with RANGE.
CURRENT ROW can be specified as both a starting and ending point.

And here is the query

SELECT id, someDate as StartDate,
LEAD((SomeDate + -1 * INTERVAL '1 day' ),1,'99991231')
 OVER (PARTITION BY id ORDER BY SomeDate) as Enddate,
  Charge,
  SUM(Charge) OVER (PARTITION BY id ORDER BY SomeDate 
     ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) 
          AS RunningTotal
  FROM test
  ORDER BY id, SomeDate

And running that query, gives us the running count as well as the end dates

id StartDate Enddate         Charge         RunningTotal
1 2012-01-01 2012-03-31 1000.0000000000 1000.0000000000
1 2012-04-01 2012-04-30 200.0000000000 1200.0000000000
1 2012-05-01 2012-05-31 300.0000000000 1500.0000000000
1 2012-06-01 9999-12-31 600.0000000000 2100.0000000000
2 2012-01-01 2011-12-31 100.0000000000 100.0000000000
2 2012-01-01 2011-13-31 500.0000000000 600.0000000000
2 2012-01-01 9999-12-31 -800.0000000000 -200.0000000000
3 2012-01-01 9999-12-31 100.0000000000 100.0000000000

Here is what it looks like if you execute the query in PGAdmin

If you don't want the last row to have the end date filled in, just omit the default value in the LEAD function. Instead of

LEAD((SomeDate + -1 * INTERVAL '1 day' ),1,'99991231')

Make it

LEAD((SomeDate + -1 * INTERVAL '1 day' ),1)

Here is the whole query again


SELECT id, someDate as StartDate,
LEAD((SomeDate + -1 * INTERVAL '1 day' ),1)
 OVER (PARTITION BY id ORDER BY SomeDate) as Enddate,
  Charge,
  SUM(Charge) OVER (PARTITION BY id ORDER BY SomeDate 
     ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) 
          AS RunningTotal
  FROM test
  ORDER BY id, SomeDate





And here is what the output looks like after we made the change







As you can see the rows for an id that doesn't have a row with a date greater than the current date will have a null end date



That is all for this post

Happy Fibonacci day, here is how to generate a Fibonacci sequence in PostgreSQL

2018-11-23T11:43:00.002-08:00

Image by Jahobr - Own work, CC0, Link

Since today is Fibonacci day I decided to to a short post about how to do generate a Fibonacci sequence in PostgreSQL. But first let's take a look at what a Fibonacci sequence actually is.

In mathematics, the Fibonacci numbers are the numbers in the following integer sequence, called the Fibonacci sequence, and characterized by the fact that every number after the first two is the sum of the two preceding ones:

1, 1, 2, 3, 5, 8, 13, 21, 34, ...

Often, especially in modern usage, the sequence is extended by one more initial term:

0, 1, 1, 2, 3, 5, 8, 13, 21, 34, ...

November 23 is celebrated as Fibonacci day because when the date is written in the mm/dd format (11/23), the digits in the date form a Fibonacci sequence: 1,1,2,3.

So here is how you can generate a Fibonacci sequence in PostgreSQL, you can do it by using s recursive table expression. Here is what it looks like if you wanted to generate the Fibonacci sequence to up to a value of 1 million

;WITH RECURSIVE Fibonacci (Prev, Next) as
(
     SELECT 0, 1
     UNION ALL
     SELECT Next, Prev + Next
     FROM Fibonacci
     WHERE Next < 1000000
)
SELECT Prev as Fibonacci
     FROM Fibonacci
     WHERE Prev < 1000000

That will generate a Fibonacci sequence that starts with 0 if you need the Fibonacci sequence to start at 1, all you have to do is replace the 1 to 0 in the first select statement

;WITH RECURSIVE Fibonacci (Prev, Next) as
(
     SELECT 1, 1
     UNION ALL
     SELECT Next, Prev + Next
     FROM Fibonacci
     WHERE Next < 1000000
)
SELECT Prev as Fibonacci
     FROM Fibonacci
     WHERE Prev < 1000000

Here is what it looks like in PGAdmin when you run the query

Happy Fibonacci day!!

Here is pretty much the same post that I created for SQL Server: Happy Fibonacci day, here is how to generate a Fibonacci sequence in SQL

This brought back memories

2018-11-20T10:08:00.001-08:00

The other day I walked past the building in the picture above and I felt a little sad. The reason I felt sad is because I have great memories about this building. When I first came to the US from the Netherlands I was amazed at how big this store was and how they had so many things on display. In Amsterdam where I arrived from, you didn’t really have a store like this. The story was named J&R, there was a J&R Music World store and a J&R Computer World store. J&R stands for the founders Joe and Rachelle Friedman.

There were several items I bought at this store

The first camera I ever owned I bought in this place, it was a Minolta but I don’t remember the model. I later sold the camera because a co-worker needed the camera while he went on vacation to Equador. So I only had it for 2 or 3 months.

A couple of years later I replaced that camera with a Canon 50E I think this Canon camera might have also been known as an Elan. The E in the model name stands for eye-control, this was an enhanced version of the 3-zone eye-controlled autofocus system that was first seen on the EOS 5 camera. This was very high tech back then.

Another item that I have fond memories about was the first MP3 player that I purchased; It was the first MP3 player that was available to buy. It was the Rio PMP-300 portable MP3 player. I believe it was around $200 and you could hold about 10 songs or so since it only came with 32 MB of memory. You could store more songs if you inserted a smartmedia card. But even then you could not store hundreds of songs. I remember encoding the songs in WMA 64k format so I could store even more songs…… lol 1^st world problems.

I bought many other things at J&R like camera lenses, filters for the camera, a tripod, RAM to upgrade the PC, I even purchased Plus! lol Plus had themes and utilities for Windows

The nice thing about J&R was that they had prices unlike 42 street photo or whatever those stores were called where you had to negotiate the price. Also unlike at the Nobody Beats The Wiz store, they did not try to sell you insurance with every item you bought.

These days I guess you would have to go to B&H to get a similar experience, just be aware that they are not open on Saturdays.