Harmless Half-Knowledge

The Next Step for AI - Full Personal Interaction Capture

2025-08-08T17:26:00.001+02:00

The open web has been and will continue to decline. The incentives to create content are further decreasing due to AI tools that answer user questions based on web search and content. Even just maintaining public access to existing content will not be obviously economical in the medium term. The text part of the web is mostly captured, images and video are under way. Since killing the open web and future content for AI training is bad, I wondered what the next big step is. The step that puts the AI data feedback loop on a sustainable path.

In my view the main goal is to capture all personal input and output via microphone, camera and computer-use observation. This would represent a step-up in data set size and quality, and get AI closer to human performance. The value add would be a cooperative AI integrated in this manner. I.e. it listens, watches and contributes through loud speakers and computer/phone use.

With this, AI platforms will cocreate and try to fully predict the article that a person has not written yet, and the book that may be written one day. Perhaps this would even yield better results as these are often unnecessarily verbose. Besides, most people won’t read the whole piece anyways. So some meta version, adapted to personal preference when viewed, seems better. Such a system requires profit sharing as an incentive or perhaps platforms can get away with just taking it. The implications for sovereignty and law are big.

When the conversations, computer-use and cameras of a large group of people (ultimately everyone) is digested, it would likely become possible to rediscover or redefine the social hierarchy. On the surface, the AI does not immediately know the relative importance of what it is looking at. It is implicit in page rank, titles, relationships, money, job descriptions, followers, social standing, contracts and labels. But it should be possible to find this relationship in the training data, and even generate a novel, perhaps more equitable, version. This could facilitate connecting skill, ideas and capital more efficiently. Again the implications are enormous.

The only company within reach of capturing all personal input and output, as well as being able to fully integrate AI cooperation is Apple. They have the devices, smart glasses on the horizon and a basic foundation with Continuity for seamless, cross-device, personal AI cooperation.

Load Local Vimrc Automatically From Parent Directories

2020-02-17T07:00:00.000+01:00

It is common to have different indention settings for different programming projects. For example:

/path/to/my/project/.vimrc:
autocmd FileType cpp setl ts=4 sts=4 sw=4 expandtab

Add the following snippet to the end of your Vimrc, so that when opening Vim, it will traverse all parent directories and open any local Vimrc file it finds. To avoid infinite recursion, traversal ends in $HOME.

    let b:dir=resolve(expand('%:p:h'))
let b:home=resolve($HOME)

    while b:dir != $HOME && b:dir != '/'
if (filereadable(b:dir.'/.vimrc'))
            execute 'source '.b:dir.'/.vimrc'
        endif
        let b:dir=fnamemodify(b:dir, ':h')
endwhile

Note that the starting directory is of the first file in your Vim command. Not the current directory, and not all directories of all files given.

I prefer simple solutions over plugins, but for completeness sake I need to mention that localrc.vim and localvimrc do the same, and likely better.

Dell XPS 7590 SSD Benchmark

2019-12-19T12:33:00.001+01:00

While I was shopping for a new laptop, I wanted to know which SSDs Dell includes in their XPS 15" models. So I called sales support. It turns out they don't know, and as of 2019, you lose warranty if you change any part. They should stop calling it CRU then! I don't know whether or not there is a seal or how they make sure you didn't tamper with the machine, or if this policy is even enforced.

So going with the included, unspecified disk was the only safe option. In my case it turned out to be a Micron 2200S NVMe disk, which is likely the same as the 2200. According to AnandTech the warranty endurance is 300 TB written, which is OK for a 1TB disk. A Samsung 970 Evo Plus has 600, my old Intel 600p has 200 TB.

It took me a while to figure out which laptop to get. In short, for me, Dell's XPS 15" cost 700€ less than the Lenovo X1 Extreme or P1 and Dell offers US international keyboards. Lenovo offers only European layouts in Germany. And you can't change them yourself in these two models.

In hindsight I'm not sure about my choice, anymore. The 4K screen is badly supported in KDE/Kubuntu. Everything is off, dialog sizes are weird, all icon sizes are weird, margins and icon placement in buttons is off. The mouse cursor switches between normal-size and half-size depending on the window you are hovering over. And if you attach an external monitor, KDE's settings offer only one global scaling factor for all screens. I.e. everything is huge if you set it 2x for the 4K internal display. The keyboard is way worse than Lenovo's and is missing the Home, End, Page-Up and Page-Down keys that I heavily use(d).

Maybe going with the Lenovo X1 Extreme or P1 with a more affordable 1080p screen and the English (EU) layout would have worked out better.

Diplom Thesis: Automatic Profile-Based Characterisation of Performance Traces for Highly Parallel Applications

2018-11-30T23:04:00.000+01:00

About four years ago I defended my diplom thesis. The title is quite a mouthful. Another way to put it is: We came up with a new way to cluster threads of execution for performance analysis.

I made the defense slides, thesis, paper and source code available.

Developing Vampir led us to the realization that scalability of performance analysis and visualization is fundamentally difficult. Information about more and more processing elements needs to be processed and displayed in a limited number of pixels on the screen.

Around 2012 we started exploring what you can do with differences between program execution traces. One thing we did was to use sequence alignment and diff algorithms to compute the difference between two function call stacks over time [1] [2]. This technique is rather expensive, but for example allows comparing the function-call-by-function-call timing of two threads of execution.

Two ways to tackle visual complexity is through folding similar executions into one [3], and e.g. by using edge bundling to cluster inter-process communication [4]. The problems these two ideas address are specific to our tools, and might not be a problem for e.g. profile-based visualizations.

Throughout the rest of the article I'll use the word process as a short form for thread, process and MPI rank.

Before I started working on my thesis, as part of an independent study course, I explored some more what kind of differences you can derive from processes and how to visualize them (slides), (report). One thing I tried was using box plots / percentiles to draw profiles on a per-process basis, instead of summing or averaging function timings across processes. The other main thing I investigated was whether to use flat function profiles, call trees, or something in between for comparing processes. One bucket in a flat function profile is a function, irrespective of which arguments it is called with, and what the call stack leading up to the call is. E.g. all calls to a matrix-matrix multiplication function are lumped into one bucket, regardless of the matrix size. A call tree profile retains the call stack for each bucket and thus provides more context. Comparing two call tree profiles is not that intuitive, and dealing with recursion is awkward. In between the two sits the idea of using the pair of functions of who-calls-whom. That provides some context, and works well with recursion. For each of these three profile types, the buckets are (1) a function, (2) a sequence of functions, and (3) a pair of functions. I also spent some time visualizing this so-called call matrix. It didn't work well. Not intuitive enough.

The important realization for my diplom thesis is that I can apply formal concept analysis to sets of function pairs to derive a kind-of hierarchy of processes. Each process is characterized by its set of function pairs (which function calls which), disregarding invocation counts and timing.

This example is a weather simulation using four MPI ranks with three threads each. All processes and threads call the top set of functions. The threads do not call any MPI or I/O functions. Process 0, on top of what processes 1-3 do also performs I/O. (The percentage displayed is the ratio of function pairs in the node to all function pairs. E.g. 37% of the pairs are shared between all processes and threads. 41% are unique to processes 1-3.)

To iron out small dissimilarities introduced by performance recording technicalities, you use the transitive closure of the function pair sets, rather than the sets themselves. Then the above example becomes three nodes on top of each other, the top-most being the threads. Thus, process 0 subsumes all other processes and threads, and process 1-3 subsume all threads. The threads are spawned by process 0-3! (The defense slides, specifically slides 16-18 and 26, have some more detail and examples on this.)

The thesis devotes much space to demonstrating the techniques on many real world examples and investigating two representatives in greater detail.

The expected complexity of creating the concept lattice (above graph) is linear in the number of processes. But the worst-case complexity is exponential, because the maximum number of nodes is. For that to happen you need a certain pattern of differences, and every process needs to have a different function pair set. In normal applications, and especially SPMD programs this never happens.

But there are three scenarios where the approach does not perform well:

The function call sequence becomes kind of random if you observe scheduling.
Load-balancing can lead to the same.
An application assigns a different cell of a space to each process. The space contains objects. As the simulation progresses objects interact and move between cells/processes. Then each process can call different functions depending on what happens within each cell. ParaDiS does something like that.

Thus the technique does not work on every code, but most. You'd need to implement fall backs to handle those gracefully.

To summarize, the technique takes the set of function pairs for each process and creates a concept lattice. This lattice represents a clustering with an attached hierarchy.

You can also use processes from multiple application runs, and not only compare processes within one run.

Clustering processes is a promising avenue that, to my knowledge, no performance analysis tool has taken, yet. Every performance tool presents a profile accumulated or averaged across all processes. With this they lump together profiles of processes that perform different work. You want to create one profile per group of processes that performs similar work. In the above example you would perhaps have one for process 0, one for process 1-3 and one for the threads. The difference between threads that share memory, and processes which communicate across nodes should be considered!

Or for example, if you think about our sequence alignment work, you don't want to align dissimilar processes, because that would result in an alignment that is just not useful. Instead you use our technique to cluster processes and then only align and compare detailed timings of similar ones.

You can also use the clustering to safe screen space by only displaying information about one representative of each group, perhaps with an indicator for timing variance.

This concludes the walkthrough of what we did. There are two things I want to mention: In the code on GitHub, I was experimenting with a more C-style C++ with templates. I don't endorse that style.

Secondly, the above technique is rather heavy-weight. You can get pretty far by just hashing the function pair set. Equal sets share the hash, unequal sets don't. Clustering done. (Then you can e.g. compute similarity between each group) This approach is easier to implement and does not have the exponential worst-case complexity. But you lose the the hierarchy. My above example talks about the process 0 vs process 1-3 vs threads relationship which is rather simplistic. A performance monitor or runtime should be able to know about this. But then again, if your analysis technique automatically yields that and more intricate relationships, that's pretty cool, too.

Costa Rica - Wildlife Refuge Laguna Urpiano

2018-03-15T19:04:00.000+01:00

Time flies. It's been a year already. I am thoroughly enjoying my job at Oak Ridge National Lab.

I got used to much of the U.S. American weirdness and other differences to home. Deep-fried sushi, deep-fried pickles, shitty bread, disgusting tap water, cars everywhere, lots of driving and so on.

July last year, three friends and I spent two weeks in Costa Rica volunteering at the sea turtle refuge Laguna Urpiano - a breathtaking experience. I'm not sure where to start. The refuge is located on the eastern side of the country without any direct access except through a canal. There is no sewage, no tap water, no electricity. The refuge has a well, runs a generator to pump water, and some solar cells that charge batteries for lighting at night, and smart phones.

On the way to the refuge

The canal

The toilets, showers and a little shack at the refuge

The housing is simple wood with a metal roof, not completely closed.

Early morning, and sometimes during the day Howler Monkeys made a lot of noise. They live high in the trees so you can't see them well. But you can definitely hear them, when they argue with their neighbors! It's not that bad, though. You get used to it.

We spent our days planting trees, helping out with work at the refuge, relaxing, drinking fresh coconut milk, and most importantly guarding one kilometer of beach every night from about 22:00 to 4:00, if I remember correctly. Napping during the day is mandatory.

Sea turtle eggs and, depending on the species, parts of the turtle are valuable. If countries like Costa Rica would not try to secure breeding grounds for sea turtles, they would go extinct. Sea turtles are very vulnerable when they lay eggs. Plus, laying eggs takes long and poachers can most of the time easily find the nests.

So we went out every night, traversing the same one kilometer every 30 minutes. When we found a turtle, we took its eggs and brought them to the refuge and built a new nest there. When we encountered a poacher, we would wait for him or her to pass. The idea is to try and be the first at the nest. The unspoken rule is that you leave the finder alone with its nest. No guns, just a machete for safety and perhaps sticks.

I can't show you any pictures of our nightly walks, and no turtles, because turtles only come ashore at night. And they use the moon for navigation. We needed to be dressed darkly and use no light except a bit of red light for writing into a notebook. No flashes allowed.

There are a few different species of sea turtles landing here, but during early July only the Leatherback visits (You can find some pictures here: (1), (2), (3)). It can grow to around 2m and 700kg. We saw just two in all our time, but during high season there are lot more. Both turtles were around 1.6m long. Impressive animals - Huge!

This is the trail of a Leatherback to its nest. The actual nest is a 20 by 20 by 40 cm hole. The rest is dug up sand to hide its exact location.

This is the remains of a fight with the sea. A Leatherback turtle decided to lay its eggs in a bad spot. The waves came in too far. If the nest / the turtle gets wet in process, the turtle aborts its mission, goes back into the sea and takes a new attempt in the next few hours or days. So we fortified the position in hopes to keep the water in check. ...But to no avail. The turtle went back without laying its eggs.

Another day we were lucky and witnessed everything. It was amazing! Such a huge animal. Paddling across the beach, inch by inch. Then it used its back fins to dig the nest hole and put the eggs in. We saw the strain on her face from the exertion. By the way, the head of such a turtle is as large as a human's. I felt deeply in awe. After that the turtle did what looks like a dance and dug up sand here and there to hide the true position of the nest.

The refuge's hatchery

The beach was littered with crabs that would hide in little holes and come out if no one is around. A bit like Whack-a-Mole where they hide when you are near. Another type of crab repeatedly rides in on a wave and tries to catch small insects like flies. Then it stands there and watches and eventually runs back into the ocean. It was a fun pass time chasing them. ... Wave comes, wave retreats, crab remains on the beach and watches around, I run in between it and the sea to try and cut it off. If that was successful, which it rarely was, I'd chase it around and try to catch it :D. And of course I'd just free it again, afterwards. I admit I injured one in the process once.

Marinera crab in its hole

A pineapple plant

We spent the last few days in Puerto Viejo, a very touristy place. Lots of surfers, lots of Germans.

The prime customer in the rescue center is the sloth, because it's too slow to cross the street quickly, and often gets electrocuted while climbing an electric pole.

These volunteers socialize with monkey children. Monkeys need that to grow up healthy.

And then it was already time to go to San Jose, and catch the flight home.

I really enjoyed my time in Costa Rica, and recommend spending a vacation there. The people at the refuge took great care of us and helped us navigate the country.

Thank you Barbara, Johnny, Emilia, Pedro, Eliseo, Alan, Maylin, Elena, Suzette and Claire!

Retirement Calculator

2018-03-09T17:12:00.000+01:00

I made my own little retirement calculator, because I was dissatisfied with how the calculators I checked do the math. And I added a bit more flexibility to play with the numbers.

It's a simple python script: (GitHub link)

    Usage: retirement-calculator [options]
    Optional arguments:
-h, --help                   Show this help message and exit
--initial-savings            <USD>
    --working-annual-return-rate <rate>
    --retired-annual-return-rate <rate>
    --inflation                  <rate>
    --working                    <years>
    --retired                  <years>
      --working-annual-savings   <USD>
    --retired-annual-withdraw    <USD>
    --verbose

The calculation works as follows:

    savings=initial_savings
    for year in range(1, working+1):
        return = savings * working_annual_return_rate
        savings += return
        savings += working_annual_savings
        savings *= (1-inflation)

And in retirement:

    for year in range(1, retired+1):
        savings -= retired_annual_withdraw
        return = savings * retired_annual_return_rate
        savings += return
        savings *= (1-inflation)

To help with picking values for annual return rates and inflation, the calculator provides some historic data.

One consideration I glossed over is taxation. Depending on your account type (brokerage, IRA, Roth IRA, ...) you do or do not pay certain taxes like capital gain tax on your realized gains or income tax on your retirement income. Keep that in mind when looking at the numbers.

The other thing I simplified is that you would normally gradually move your investments towards safer options as you get closer to your retirement date. The calculator assumes a hard switch in annual return rate at the start of retirement.

Note that all numbers are in today's US Dollars. That also means that the calculator assumes you increase your annual savings to match the rate of inflation.

For example, if you work for 35 years, save $6000 a year, assume an annual return rate of 7% during work life and 5% during retirement and 3% annual rate of inflation, you end up with $411020 of today's US Dollars, and an initial annual return of $19351 at the start of retirement. With that you can roughly withdraw $17000 per year for 30 years.

So that's no good.

The difficult part is estimating future returns on your investments.

Playing with the numbers, I'd say if you want to be safe, saving $10000 to $15000 per year seems to be about right. The more the better.

Here is a sample output: (link)

Great Smoky Mountains / Cades Cove Hike

2017-05-18T07:00:00.000+02:00

The Great Smoky Mountains are part of the Appalachians - a system of mountains in the east of the U.S., ranging from the bottom all the way to the top. Notably in the Smokies, you can oftentimes see wild black bears. Bears aside, there are a number of dangerous animals living in Tennessee, e.g. Rattlesnakes, Copperheads and venomous spiders.

The following photos are from a hike on the Anthony Creek Trail in Cades Cove. At the end of the trail we met a guy who hikes the whole Appalachian Trail from Georgia all the way up to Maine. That is 3500km and takes five to seven months - Holy shit!

On our way

Lots of fog and rain on our way up

Reached the end of the trail

A colorful tree in the middle of green, brown and gray

A creepy millipede - Xystodesmidae / Apheloria perhaps?

At the visitor center - a friend wanted to collect the stamp for this national park.
Her goal is to visit each national park and collect all 59 stamps.

Dark Souls 1 Photo Tour

2017-05-08T07:00:00.000+02:00

After I've enjoyed all content Dark Souls 2 has to offer (photo tour), I went back to Lordran. Here's a selection of photos I took during my stay.

But first some even older photos from 2014.

Let me introduce my Wanderer, sporting the Uchigatana and the Grass Crest Shield. I stuck to the basic wanderer armor as much as possible, because it's pretty and light.

You take great care setting up your character, and then spend most of the time
as a beef jerky zombie. *sigh*

Up there lives the giant archer Hawkeye Gough who can shoot down dragons.


My friend Solaire of Astora - One of the most popular characters in Dark Souls. He spends a lot of time looking at the sun and musing.

The Tomb of Giants - The Skull Lantern lights the way in this godforsaken place.

Blacksmith Vamos

Now for the new photos after my return:

I am in The Duke's Archives. Alright, bring it on!

Enjoying the cutscene

Seath the Scaleless who betrayed his fellow dragons
and thus helped end their rule and bring about the age of fire.

Siegmeyer of Catarina, a.k.a. The Onion Knight

I can be an onion, too.

My tour guide Darkstalker Kaathe goofs around

The Painted World of Ariamis is guarded by a beautiful dragon+human-hybrid woman.

Tis but a scratch.

Since I'm not perfect at the game, I had to learn some magic, get heavier armors
and specialized gear.

The Demon Ruins

Solaire joins for some jolly cooperation. \[T]/

The Ash Lake - It's far underground and has a beach and a sky...

The Stone Dragon - He initiates you into the Path of The Dragon covenant.
In this covenant you basically fight other players in return for shiny gear.

Eventually, I gathered all boss souls and opened the gate to the last area ...

... The Kiln of the First Flame

The last boss took me a week or two. Good dammit!
Finally, with the help of the strongest and heaviest armor,
I was able to beat him. Not pretty, but it worked.

Like so many other players before you get to decide whether you offer your soul
to rekindle the flame and thus extend the age of fire,
or bring about the ultimately inevitable - the age of darkness.

In the end it doesn't matter. Nothing you did matters.

And thus repeats the cycle.

New Game Plus is over, and NG++ starts. Everything repeats, except you keep your character and every enemy is stronger.

Once the tutorial is done, a giant crow carries you away.

Enjoy the scenery of Lordran while you can.

This is your home: the Firelink Shrine. The guy on the right already lost his mind.
Most characters in Dark Souls are crazy or lose it later on,
because the world is ruthless and there is no hope.

This Gargoyle guards one of two Bells of Awakening.

Ringing the Bell of Awakening.

Anor Londo

"Amazing chest ahead" - Gwynevere, Princess of Sunlight

Seath once more

Praise the sun \[T]/

In a Galaxy Far Far Away

2017-04-08T19:40:00.000+02:00

As many of you already know, at the beginning of the year I joined Oak Ridge National Laboratory as a Research Associate. My job is to use my knowledge in HPC performance analysis to enable scientific codes (like these) to run better on Titan - the fastest open-science super computer of the western world and its successor Summit (POWER9+NVIDIA Volta). Needless to say that this is super-exciting work!

As a result, I now live in Knoxville/Tennessee and there is so much to talk about :). This is not a coherent post. More like a fact and opinion rapid-fire.

The flag of Tennessee, where it is on the map, and the ORNL Logo

Everything feels around 1.5x as expensive as in Germany. Around what I would expect if the exchange rate to Euro would have stayed where it initially was. Gas costs 53¢/l. Electricity costs 9.8¢/kWh. Let that sink in for a second - Electricity and gas are practically dirt-cheap. A full tank costs me $23. No wonder people drive these giant-ass pickup trucks.

Houses are generally cheaply made - starting at ~$50k. In my apartment the air conditioning system both cools and heats. This combination of cheap housing and super-inefficient heating resulted in my first two month's electricity bills being around $40 (~400kWh). And this has to be just heating and maybe the fridge (It's a biiiig fridge). Although electricity is cheaper, I managed to pay more than a single-person household would pay in Germany. Electricity is so cheap people don't even bother getting LED light bulbs.

Wow, just wow.

Meat, especially beef is a lot cheaper. You can get a decent steak for $15 per kg. In general beef tastes a lot lot better than what you can get in Germany. It's seriously amazing. Beans and rice are also cheap. Ben & Jerry's costs $4-5, which is a blessing :).

A gigantic T-bone steak

Dinner time <3

My phone bill is around $30 for 20 minutes, 30 messages and 550MB, and Comcast wants $70 for 25/5MBit. Comcast has the worst customer service I have ever seen. I spent many attempts and a total of 2-3 hours to get two things sorted out. It's actually ridiculous how unhelpful their customer support is.

The typical rent in Oak Ridge/Knoxville ranges between $500 and $1000 for a normal sized flat.

Spatially, everything here is very very spread out. Knoxville has a population of 185k on roughly Dresden's area, and Knox County has 430k on five times that area. Oak Ridge has 30k people on 3/4ths the area of Dresden. In order to go downtown I need to drive about 8km. To the next mall I need somewhere between 2 and 8km depending on which. The only thing where I can really walk to is the gas station down the road. Sidewalks are rare, almost non-existent. Only downtown is it possible to get anywhere without a car. That makes having a car essential. This also means you cannot run, bike or take a decent walk from your door step, but rather have to drive somewhere and then go from there. It's annoying having to drive, go for a run, and then drive home all sweaty.

So I had to buy a car. The best option turned out to be to just buy it from a friend who left the country at the time I arrived. It's a shiny 2015 Ford Focus Sedan 6-shift manual.

When I arrived in the USA, I used a rental car for while. During that time I came to like automatic transmission. Sadly this one is a manual. Well, at least no one will be able to steal it - although.

I think Focus sedans don't exist in Germany. Overall there are more big cars on the road here and most of them are sedans and pickup trucks. Hatchbacks are very rare. Almost no one drives small cars. Very rarely do you see a Smart, a Fiesta or a Golf.

I bought a dash cam to play around with and for insurance purposes. People here are bad drivers: No signaling, random lane switching, overtaking on the right. Something like the TÜV does not exist. There are cars driving around where you are surprised they are able to drive at all. This also means there is no emission checking. I saw couple of cars that emitted pitch-black fumes. What the hell. Getting a driver's license is very easy, and costs just $30. Many don't even have car insurance. Phoning while driving is allowed. Texting is not.

In Tennessee, you can film anywhere where people have no reasonable expectation of privacy. Which is (almost?) everywhere in public. Everywhere else you are allowed to record conversations of where you have at least the consent of one involved party, for example yourself.

When you go shopping, everything is put into bags for you by the cashier. Bags bags bags. One notable exception is Aldi. Here's a picture of the result of my first shopping tour in the US.

No one recycles. In order to recycle you need to drive to the next recycling center - 7km for me. It's just a simple place with big containers for each kind of waste. Before I began recycling, I created at least four times the amount of trash I generated in Germany, which is crazy. Separating biodegradable waste is not a thing at all.

Many stores are open every day and for way longer than in Germany. For example at Walmart you can go shopping 24/7.

Tennessee sees extremely quick weather changes, I think because it's continental and the mountain ranges are vertical so that weather coming in from north and south passes through quickly. One day it's 0°C, the next day it's 10°C and a few days later it can be more than 20°C. And then suddenly a thunderstorm with 1cm hails lasting for just one hour shows up. Ughh. I feel like I have seen more extreme weather in four months than my hole life.

Here's nine people blowing leafs around the apartment complex towards their truck. I would expect rakes, but no.

Another peculiarity is beer. The old opinion that American beer is bad does not hold anymore. Actually, I think it's overall better than in Germany. Mainly because the variety is so large. There are many small breweries creating all kinds of beers - IPAs, Ales, Stouts, Lagers, Pilseners. Their Pilseners are terrible in comparison to ours, though. One big downside is that I get headaches very quickly from their stuff. Not sure what exactly causes this. Perhaps it's the added sugar in some of them. The price for beer starts at around $1-1.5 for a 350ml bottle of industrial beer. A beer at a bar is usually between $4 and $6.

A bazillion beers on tap. Every time a barrel is emptied,
they add a different beer. Great!

They ship this to here!

Create your own six-pack for $10 at Kroger.

The German/Austrian beers at Bearden Beer Market.

Trivia is a big thing around here. Lots of bars do it on a weekly basis. Our moderator Will at Hops and Hollers does an awesome job at it. Trivia is awesome! You have to collectively find the right answer, and betting-rounds or be-as-close-as-you-can-to-the-correct-answer increase tension and fun. Since I'm here we managed to win twice, and perhaps we will make it to a second-tier tournament later this year. Our strong suits are science, music, movies, history, religion, alcohol, geography, everything German / European. We are reasonable good at sports and suck a lot at Celebrities, TV and commercials.

Imperial units. Some call them freedom units or retard units. Only a handful of countries still use them. I got used to them more quickly than I expected. You only convert when you need to be exact. Most of the time you don't convert at all. On the road 10 miles is far, a quarter mile is soon, 1000 feet is sooner and 100 feet is very soon. An ounce is 30 grams. A pound is half a kilogram. A quarter pound is 100g, 12 fluid ounces is a can, a freedom pint is 16floz which is around 500ml. Yes, an American pint is really smaller than an imperial pint. A gallon is four liters.

While I'm here I have the goal to try as much local things as possible. I tried Pop Tarts, Twinkies, steak, philly cheese steak sandwich, mac & cheese, burger, hot dog, pizza, salsa, tortillas, tacos, quesadillas, chicken wings, and all those food chains like Chipotle, Five Guys, Blaze Pizza, Wendy's etc. There is actually quite a few things that are super-tasty!

Twinkies do have an expiration date!

Among other things I also want to see a rocket launch, go to a republican rally and to the shooting range. That's going to be interesting!

Since the election of Donald Trump politics has been a big shitfest. While Germany wrangles over sharing passport data with the police and meta data retention, things like these happen here:

Immigration-limiting executive order
Bill that allows ISPs to sell your browsing history
Companies might soon be able to force genetic tests on employees and share the results with third parties.
Trump's budget proposal would cut science by $50B and increase military spending from $550B to $600B.
JPL scientist had to unlock his work phone at the airport. The aftermath is a big discussion on how to travel with electronics to and from the USA.
Executive order to remove the privacy-rights of foreigners living in the US
Executive order to ignore climate change in favor of helping coal and oil

I hope that voters pick up on these things and react accordingly.

Oak Ridge National Laboratory, short the lab, is a very interesting place to work at. It is located on a 40km² patch of land in the middle of nowhere surrounded by mountain ranges. To get to it you need to drive at least 10km. My commute from Knoxville is about 30km. The nearest airport is 45km away.

Historically it is one of the sites created for the Manhattan Project. More specifically, the lab was tasked with creating large amounts of weapons-grade plutonium.

During this time the city of Oak Ridge has been established.

Today the lab still has the old graphite reactor opened for tourists. Besides that it has a number of leading-edge science facilities. Some of them are:

The Spallation Neutron Source (SNS), a giant nuclear research reactor used for neutron imaging and radiation experiments by hundreds of scientists from all over the world
The High Flux Isotope Reactor (HFIR), a different kind of research reactor
The infamous Building 5300 which is right next to my workplace - Classified things are going on in there. Read the Wikipedia article for more info.
The Oak Ridge Leadship Computing Facility (OLCF)
And many more

The OLCF houses Titan, the #1 civil supercomputer of the western world. In the Top500 there are two Chinese machines that score higher, but that's it. Titan consists of 200 cabinets, 18,000 nodes, 18,000 NVIDIA Kepler cards and 300,000 AMD Opteron cores.

The successor to it is Summit, which will be built by IBM and will contain POWER9 processors, NVIDIA Volta cards and NVRAM. It will deliver north of 100 petaFLOPS. Before Summit starts arriving later this year, right now we have a test setup consisting of POWER8 + P100 cards and local NVMe storage to gain experience with. Shiny!

The lab has a staff of around 4000 people, plus 3000 guest researchers, and 500 interns annually from all around the world. It is a very diverse work place. I could not find numbers, but I would guess that at least a third are foreigners. I know people from Belgium, China, Egypt, England, France, Germany, India, Kyrgystan, South Africa and Spain. It feels like almost everyone has a PhD, and people are generally super-smart and relaxed. I love it.

One of the main tasks of the national labs in the USA is the maintenance of the nuclear arsenal. That includes for example simulating the decay of the weapons.

The nature of interactions at the lab is a bit different from what I am used to. Oftentimes people don't directly answer/admit something. There is a kind of double-speak depending on the subject. I can't really put a finger on it or want to give examples. It's just how it feels. I think this is a product of the type of work some people are doing, the things some people know/suspect, the fact that you should assume to be watched, and the fact that suing people is a national sport. The amount of ass-covering going on is substantial, not just at the lab, but in general. For example there is a banner in my building thanking people for millions of safe work hours. And packaging of food or hot beverages often tells you not to be an idiot.

The lab is used both by the Department of Energy (DOE) and Department of Defense (DOD). The DOD is the parent agency of the NSA.

I don't know how it exactly works, but I think the DOE is a rather civil branch whereas the DOD is the military. One of the tasks of the DOE is to maintain and control the nuclear stockpile together with the DOD, so that not only one department has control over it.

The median income per person in the USA is around $30k, the median household is $50k. Tennessee is a bit below that. With a master's degree I am at more than twice the per-person-median and in the top 20%. I feel the income distribution is a bit worse than in Germany, especially towards the lower end. I feel that lab employees are overall fairly rich. Rumor has it that better positions pay way way better than e.g. higher positions in German public service. It's run like a private company.

To conclude this post, here are a few photos for you to enjoy:

A panoramic view of a few houses and the Interstate 40

Lake Shore Park is nice for running.

On our way to Ober Gatlingburg to go skiing

There is a Disney store in West Town Mall

It consists mainly of Star Wars merchandise and princess costumes.

Pi Day!

Authentic German cuisine. Haven't been there, yet.

A muskrat minding its business

The lab geese are having babies! Update: The nest is gone. Investigation pending.

I wonder whether these two have names. If not I will name them.
Perhaps they should have badges like any other employee.

A bit of Saxony far from home. Thank you Aldi.

Freedom sushi: Deep-fried, mayo-based sauce, creem cheese.
A friend ordered tea, and the first thing offered was sweet tea, haha.

Someone at the Lab drives this bad boy

I hope you are having a great time at home. So far I'm having lots of fun here. Make sure to hit me up if you are nearby. I'll try to be at the Super Computing Conference later this year. Hopefully, I'll see you there. Bye!

Intel SSD 600p (1TB) Luks+Ext4 Encrypted Disk Performance

2017-02-13T22:13:00.000+01:00

I got myself a new toy - The Intel SSD 600p 1TB M.2 NVMe disk. Before I bought it, I tried to figure out how big the performance penalty for using full disk encryption would approximately be. But I wasn't able to find much on this topic. Now that I set it up, I want to share some benchmarks of how an encrypted LUKS+Ext4 partition performs in comparison to just using Ext4.

All benchmark runs were made with GNOME Disks on a Lenovo Thinkpad T460s with a Core i5-6200U CPU running Ubuntu 16.10 Yakkety Yak (Kernel 4.8.0-37-generic x86_64) and KDE. The encrypted partition was set up via GNOME Disks.

LUKS+Ext4

1 MiB Block Size

10 MiB Block Size

100 MiB Block Size

1000 MiB Block Size

The 1000 MiB block size attempts were inconsistent. After a minute of waiting before doing another run, I got the above result. Many look like the two runs below. It is either a temperature or caching effect I guess.

1000 MiB #2

1000 MiB #3

Ext4 Only

1 MiB Block Size

10 MiB Block Size

100 MiB Block Size

1000 MiB Block Size

Evaluation

Just reading/writing data at 1.8/0.6 GB per second takes CPU time, regardless of whether encryption is enabled or not
LUKS+Ext4 incurs signifcant extra CPU work over just using Ext4. Notice that the CPU uses all four threads for the encrypted partition, but just one or two threads for the unencrypted one.
Access time according to GNOME disks is similar. The difference is less than a factor of 2 in all cases.
1 MiB Block Size: Encryption incurs a bandwidth penalty of factor 2 to 4.
10 MiB: There is a read slow down factor of 2 with encryption. Writing is not much slower.
100 MiB: The encrypted drive alternates between 1.7 GB/s and 1.0 GB/s read speed for some reason. Writing looks OK.
1000 MiB: You get almost full speed.

Race Report: 19ter Lauf in den Frühling

2016-05-30T06:00:00.000+02:00

Am 6. Mai war es soweit, der Lauf in den Frühling stand an. Laut Trainingsplan sollten die 44 Minuten für 10km möglich sein. Andererseits ist eine 4:24min/km pace ziemlich hart.

Nachdem die Kleinen jeweils 800m und 2km gelaufen, sind gehen die Großen an den Start. Ich war schon ein bisschen nervös. Vorsichtshalber habe ich mich weiter vorn postiert damit ich dann nicht so viel überholen muss. Kurz vor dem Start erfuhr ich noch dass es keine Kilometermarken gibt. Das bedeutet ich werde die ganze Zeit nur eine wage Ahnung haben wie schnell oder langsam ich bin. Ärgerlich.

Los geht's.

Die ersten paar Kilometer sind sehr flott und gehen gut von der Hand. Das Höhenprofil in Kombination mit dem Tempo macht den Lauf anstrengend.

Die ersten fünf Kilometer gehen zügig vorüber. Nach dem ersten Wendepunkt (Spitze des Berges) auf dem Weg zurück treffe ich die Kollegen recht spät. Ich bin wohl ziemlich ziemlich schnell.

Ab Kilometer 7 geht mir dann langsam die Puste aus. Ein bisschen langsamer machen. Ab dann wird es stückweise immer anstrengender und ich werde gefühlt immer langsamer. Wie langsam? Keine Ahnung. Wohl immer noch schneller als 5:00min/km.

Ungefähr bei Kilometer 8 kann ich in der Ferne das Ziel sehen. Aber dort muss jeder nochmal zum zweiten Wendepunkt abbiegen. Der Körper würde aber gern aufhören. Puh. Durchhalten. Auf den letzten drei Kilometern haben mich circa zehn Läufer überholt :(.

500m vor dem Finish steht fest: Ich schaffe die 44:00. Entspannung setzt ein und das Tempo sinkt noch ein wenig. Im Ziel sind es dann 43:12 (4:19min/km, 13.9km/h) Wohoo \o/. Die Platzierung ist auch ordentlich, sodass ich als einer der ersten 100 im Ziel ein T-Shirt gewinne.

Die drei Kollegen kommen bei 47 bis 50 Minuten ins Ziel. Leider haben nur zwei von uns ein T-Shirt gewonnen. Ist aber auch nicht schlimm, weil nur Werbung in textmarker-grün drauf ist. Nichtmal "Lauf in den Frühling" oder etwas ähnliches steht drauf.

Danach geht's ins Hains noch eine Stunde planschen. Das war im Startpreis inbegriffen. Super! Zwei neue Rutschen gilt es auszuprobieren. Eine rutscht man in einem Luftring herunter. Die andere ist eine Paarrutsche mit Zeitnahme, wo zwischen den beiden Bahnen teilweise nur eine transparente Wand ist sodass man den Anderen sieht. Yeah! Vier 30+-Männer haben lauthals ihren Spaß.

Alles ein allem ein Top-Lauf, von der Orga, den Bedingungen, und der eigenen Leistung her. Und die Startgebühr war so gering dass mit dem freien Eintritt und dem T-Shirt es sich sogar gelohnt hat ;).

Nächstes Jahr gerne wieder!

----

Dieses mal hat mich mein Puls und meine Geschwindigkeit auf der Strecke wirklich interessiert. Aber bisher laufe ich zum Wettkampf nur mit der Vivosmart HR.

Die Kurve muss man mit Vorsicht genießen, weil der optische Heartrate-Sensor nicht so gut ist - vor allem bei Tempo- und Temperaturvariation. Die Zahlen bis Minute 9 sind zum Beispiel komplett falsch. Die maxHR von 188 bei ca 21 Minuten stimmt wohl und dass ich bis Kilometer 7 mit immer über 180 ordentlich gelitten habe bzw. zu schnell gelaufen bin klingt auch richtig.

Race Report: 19ter Oberelbe-Marathon

2016-05-17T06:00:00.000+02:00

Weil ich beim Citylauf Blut geleckt habe, besorgte ich mir darauf hin einen Trainingsplan für 10km in 44 Minuten und fing an zu trainieren.

Fünf Wochen später am 24. April fand dann der 19te Oberelbe-Marathon statt. Die Kollegen waren schon länger für den Halbmarathon angemeldet und ich habe mich dann entschieden mitzumachen als ich im Training die Sicherheit gewonnen hatte dass meine Knie und Sehnen die Strecke auch wirklich aushalten.

Kurz vor dem Rennen haben wir die persönlichen Ziele gesteckt und festgelegt dass jeder sein eigenes Tempo läuft, weil unsere Geschwindigkeiten zu unterschiedlich sind.

Los geht's mit 2100 anderen Startern von Pirna nach Dresden. Ich laufe wieder relativ langsam los um mich nicht schon am Anfang zu übernehmen. 21.1km sind nämlich ziemlich ziemlich lang.

Nach 5 Kilometern brechen die Ersten ein und die Überholrate steigt.

Nach 6 Kilometern kassiere ich den 1:45:00 pacemaker - Wohoo, super!

Da ich bisher selten lange Strecken gelaufen bin, hatte ich vor allem Angst dass irgendwann die Energie alle ist und ich einbreche. Im Training hatte ich mal ein Tief nach 13 Kilometern. Puls und Atmung waren immer OK, aber davor hatte ich ein bisschen Angst.

Auf den folgenden vielen Kilometern genieße ich die Aussicht, überhole viel und freue mich über die zahlreichen Familien, Omas und Opas mit Plakaten, Rasseln und Trillerpfeifen die uns anfeuern. Manche haben mich sogar persönlich angefeuert weil ja mein Name auf der Startnummer steht. FeelsGoodMan.

Nach etwa 12 Kilometern fiel die Entscheidung nicht mehr auf die Uhr zu schauen. Das Umrechnen auf Gesamtzeit und pace ist ad hoc zu schwierig und es stört den Rhythmus - Einfach laufen.

Das Tief blieb glücklicherweise aus. Vielleicht hat es auch an den zwei Gels gelegen die ich mir nach 10 und 15 Kilometern reingezogen habe. Ekelhaftes Zeug. Bah!

Nervig waren noch etwas die Verpflegungsstationen. Bei 12km/h einen Becher zu greifen und zu trinken ist schwer. Da geht der Großteil einfach vorbei am Mund. Danach war man aus dem Rhythmus und musste erst einmal wieder den Puls und die Atmung unter Kontrolle bekommen. Ging aber irgendwie.

Die letzten Kilometer haben sich in die Länge gezogen, versuchte Temposteigerungen auf den letzten zwei Kilometern wurden schnell wieder abgebrochen, und auch der Zielsprint wurde nach fünf Sekunden abgesagt - Puh, war ich kaputt.

Als ich über dem Zielbereich die Bruttozeit von 1:39:xy sehe freue ich mich riesig - 1:45:00 war das ursprüngliche Ziel.

Die Nettozeit betrug am Ende 1:38:29 (4:40min/km, 12.9km/h). Das war also fast die gleiche Geschwindigkeit wie bei den 10km vor 5 Wochen - Wahnsinn, gigantisch gut.

Platz 251 von 1500 Männern. \o/

Die Kollegen habe ihre eigenen Rekorde auch alle verbessert, außer Ronny. Der hatte Knie und musste deswegen langsam machen. Weil er beim Citylauf so locker dabei war, hatte ich mich schon darauf gefreut ihm dieses mal Konkurrenz zu machen und ihn leiden zu sehen. Aber daraus wurde nichts. Najo, beim nächsten mal dann.

Das Wetter während des Laufs war übrigens scheußlich: Gegenwind an der Elbe, Regen, Hagel und Sonne im Wechsel.

Alles in allem war das Event super und alle sind Happy mit ihrem Ergebnis.

Danke an alle am Lauf beteiligten!

Race Report: 26ter Citylauf Dresden

2016-05-09T06:00:00.000+02:00

Kurz vorm 19. März haben mich die Kollegen überredet beim 26ten Citylauf Dresden mitzumachen. Da ich ab und an laufe war von vornherein klar, dass ich die 10km zumindest schaffe. Die große Frage war: Wie schnell?. Letztendlich haben wir uns (Matthias, Ronny und Ronny) vorgenommen unter 50 Minuten (5min/km) zu bleiben.

Auf den ersten Kilometern lief es sehr gut. Nach kurzer Zeit hatte ich schon eine Minute Vorsprung auf die 50 Minuten herausgelaufen.

Bei Kilometer 5 habe ich dann Matthias und Ronny, die schneller angelaufen sind, eingeholt. Ronny hängte sich dann an mich. Oder ist zwischen uns gependelt? Er sah jedenfalls viel viel zu entspannt aus.

Die letzten drei Kilometer waren eine ziemliche Quälerei. Im Zielsprint hat mich Ronny noch abgehängt und sah dabei immer noch entspannt aus.

Letztendlich waren es 46:24 (4:39min/km, 12,9km/h) Nettozeit - Ein Spitzenergebnis für mich.

Für ein Event mit 2600 Läufern (1500 auf der 10km Strecke) war alles überraschend entspannt.

Alles in allem war meine erste Lauferfahrung super :)!

Danke an Matthias und Ronny und alle am Lauf Beteiligten!

Thank you From Software

2016-05-02T06:00:00.000+02:00

I recently finished Dark Souls 2 NG+4 and acquired all achievements.

For this occasion, I just wanted to say thank you and share some images of my time in Drangleic.

It's been a long journey

Just beat Nashandra and Aldia

Proof

Proof #2

My level 313 Wanderer Aria

Where it all started: The Old Firekeepers

Home

Home #2

Home #3

Home #4

Waiting for PvP...

Sun Bros for Life

Dragon Shrine

Heide's Tower of Flame

Hello darkness my old friend: The Ancient Dragon

I'm about as big as his foot

Praise the Sun \[T]/

Next up I'll be going back to Dark Souls 1 and continue my Level 142 character in NG+.

If you've played these games for long enough, they are more soothing and relaxing than stressful. I love coming back to Drangleic and Lordran to calm down after a long day of work.

Thank you From Software

Error with Scripting Vim in Python - Ubuntu Xenial

2016-03-10T18:21:00.000+01:00

While updating to Ubuntu 16.04 (Xenial Xerus) my .vimrc stopped working with the following error message:

    vim.error: Vim(python):E319: Sorry, the command is not available
    in this version: python << endpython

After a bit of wrestling I found out that you have to replace

    python << endpython

with

    python3 << endpython

The big tell was that $ vim --version yielded, among other things, -python and +python3.

OTF2 Knowledge: How to Match MPI Messages

2015-07-26T10:00:00.000+02:00

OTF2 (part of Score-P) has a terrible reader interface. This post explains one little part of boilerplate you need to write in order to process MPI messages.

To properly process MPI send and receive, you need to determine which send belongs to which receive. This in conjunction with the previous two posts (1, 2) enables you to generate one list of messages to be used in your tool, subsequently.

Note that the following explanation also applies to OTF and likely to processing MPI send and receive events in general.

Formally, given:

struct SentMessage {

sender, receiver, time, communicator, length, tag }

struct ReceivedMessage {

        receiver, sender, time, communicator, length, tag }

    list<SentMessage>     sentMessages
    list<ReceivedMessage> receivedMessages

Where both sent and received messages are ordered according to when these operations where issued, we want to determine:

struct Message { sender, receiver, time, duration, length }

list<Message> messages

First, we transform the received messages into another data structure:

struct Key { sender, receiver, communicator, tag }

map<Key, queue<ReceivedMessage>> receivedMessagesMap

Second, we iterate over all sent messages and index into the received messages map, to find the matching receive.

For each sent message

Use its sender, receiver, communicator and tag fields to index into receivedMessagesMap and retrieve the queue
Is there an item in the queue? Yes:

The first item in the queue is the receive that matches to this send
Append the matched message to messages. Duration is the difference between sent and receive time.
Dequeue this item
If duration ≤ 0: Warning: A message should be received later than it has been sent. We have undefined/infinite or negative bandwidth.
If send length > receive length: Warning: A sent message is not allowed to be larger than the receive assumes.

Warning: Missing receive

Print out a condensed version of the above warnings

We use one queue per sender, receiver, communicator and tag, because a send belongs to a receive only if all of the four coincide.

Recording the warnings/errors makes sense, because MPI libraries are less strict than the standard. For example, if you send messages and never receive them and just exit the program, neither the MPI library nor Score-P might care and thus you have missing receives. I can confirm that this happens sometimes. Therefore, you should not disallow missing receives.

Also, be aware that durations will likely be zero or negative from time to time. Time depends on the clocks used. Clocks across different nodes/processes can yield values that imply zero or negative durations. Score-P does not sanitize time values. Thus, it is left to the tools developer to decide how to handle this situation.

With these three posts you can now properly process MPI messages, for example to compute correct bandwidths, or to visualize them:

Happy coding!

OTF2 Knowledge: How to Process Non-Blocking MPI Messages

2015-07-05T16:18:00.001+02:00

OTF2 (part of the Score-P) has a terrible reader interface. This post attempts to explain one part of boilerplate you need to write in order to process MPI messages.

If your tool processes MPI messages, it does not suffice to only consider MpiSend and MpiRecv records. Applications using MPI can also use non-blocking send and receive operations. Mixing blocking and non-blocking operations is possible as well. In order to handle MPI messages correctly, you need to consider all of the following records:

    OTF2_CallbackCode handleMpiSend(OTF2_LocationRef sender,
        OTF2_TimeStamp time, void* userData, OTF2_AttributeList* a,
        uint32_t receiver, OTF2_CommRef com, uint32_t tag,
uint64_t length)
    OTF2_CallbackCode handleMpiIsend(OTF2_LocationRef sender,
        OTF2_TimeStamp time, void *userData, OTF2_AttributeList *a,
uint32_t receiver, OTF2_CommRef com, uint32_t tag,
        uint64_t length, uint64_t requestId)
    OTF2_CallbackCode handleMpiIsendComplete(
        OTF2_LocationRef sender, OTF2_TimeStamp time,
        void *userData, OTF2_AttributeList *a, uint64_t requestId)
    OTF2_CallbackCode handleMpiRecv(OTF2_LocationRef receiver,
        OTF2_TimeStamp time, void* userData, OTF2_AttributeList* a,
        uint32_t sender, OTF2_CommRef com, uint32_t tag,
        uint64_t length)
    OTF2_CallbackCode handleMpiIrecv(OTF2_LocationRef receiver,
        OTF2_TimeStamp time, void *userData, OTF2_AttributeList *a,
        uint32_t sender, OTF2_CommRef com, uint32_t tag,
        uint64_t length, uint64_t requestId)
    OTF2_CallbackCode handleMpiIrecvRequest(
        OTF2_LocationRef receiver, OTF2_TimeStamp time,
        void *userData, OTF2_AttributeList *a, uint64_t requestId)
    OTF2_CallbackCode handleMpiRequestCancelled(
        OTF2_LocationRef location, OTF2_TimeStamp time,
        void *userData, OTF2_AttributeList *a, uint64_t requestId)

This post explains how to translate non-blocking send and receive records to normal/blocking send and receives, so that your tool can subsequently process all types of messages in a homogenous way.

In a previous post I explained a detail you need know for this to work.

Explaining the Involved Records

Send: Is issued when MPI_Send is called

Isend: Is issued when MPI_Isend is called

IsendComplete: Is issued when an MPI_Wait, MPI_Test or a similar function confirms that the Isend operation has been completed

Receive: Is issued when MPI_Recv is called

IreceiveRequest:

Is issued when MPI_Irecv is called
Similar to the Isend record, except you don't yet know the tag, communicator and length of the to-be-received message, because of possible wildcards for tags and communicators in such requests

Ireceive:

Issued when an MPI_Wait, MPI_Test or a similar function confirms that the Ireceive operation has been completed
Similar to IsendComplete, except it contains the parameters of the received message, whereas Isend itself (not the complete) contains them

RequestCancelled: Can cancel Isends and IreceiveRequests, and other maybe not-recorded requests.

The naming scheme of these records is a bit confusing. A more consistent scheme would have been: Isend, IsendComplete, Ireceive (=IreceiveRequest here), IreceiveComplete (=Ireceive here).

Data Structures

Send/Receive

struct SentMessage {

time, sender, receiver, communicator, length, tag }

struct ReceivedMessage {

time, receiver, sender , communicator, length, tag }

Non-blocking

struct Isend { time, sender, receiver, com, length,

tag, requestId, queue<SentMessage> blockedSends }

struct IreceiveRequest { requestId,

queue<ReceivedMessage> blockedReceives }

map<process, queue<Isend>> isends

map<process, queue<IreceiveRequest>> ireceiveRequests

The Algorithm

Send

If isends for this process is empty
- Record the sent message
Otherwise
- Append this Send to the latest isends[sender]'s blockedSends

Explanation: To preserve the correct ordering of messages, previously issued Isends have to be processes before this Send. We can only process Isends that are completed, because they might be cancelled subsequently. We therefore enqueue this Send until all previous Isends are processed.

Isend

Append this Isend to isends[sender] with an empty queue

IsendComplete

Search for a matching Isend on this process
If it matches the first in the queue
- Record the sent message (time is Isend's time)
- For each blocked Send in the attached queue
  - Record the sent message
Otherwise
- Append this completed Send to the previous Isend's blocked queue
- Append the blocked queue of the completed Send to the previous Isend's queue as well
remove this entry from isends[sender]

Explanation: An Isend has been completed. Therefore, we have a succesfully sent message that we can record unless there are previously issued, uncompleted Isends (similar to when Send happens). If we completed the earliest remaining Isend, we can now process all messages that have been blocked by it. If we completed an Isend that has previous other Isends, then we complete this Send and enqueue everything to the previous Isend's queue, because we can only process these messages when this previous Isend has been completed.

The receive records will be handled similarly, with some slight modifications due to differences in whether information is known during the request start or completion.

Receive

If ireceiveRequests for this process is empty
- Record the received message
Otherwise
- Append this Receive to the latest ireceiveRequests[receiver]'s blockedReceives

IreceiveRequest

Append the request to ireceiveRequest[receiver] with an empty queue.

Ireceive

Search for a matching IreceiveRequest on this process
If it matches the first in the queue
- Record the received message (time is Ireceive's time)
- For each blocked Receive in the attached queue
  - Record the received message
Otherwise
- Append this completed Receive to the previous IreceiveRequest's blocked queue
- Append the queue of the completed Receive to the previous request's queue as well
Remove this entry from ireceiveRequests[receiver]

RequestCancelled

If it matches an Isend's requestId

Handle the same as IsendComplete, but without recording this send

If it matches an IreceiveRequest's requestId

Handle the same as Ireceive, but without recording this receive

It should never be both, but can be neither

Wrap Up

A sent message has the timestamp of the initial Isend. A receive has the timestamp of the completed Ireceive. Thus, we take the timestamp of the earliest possible sending moment and the latest possible receiving moment, because the message is actually transmitted sometime during this interval. This is as accurate as it gets in OTF2. Because of this, IreceiveRequest's and IsendComplete's timestamps are ignored. This also means that the order in which you process the records is not necessarily chronological for receives, but it is for sends.

After applying the above algorithm you have a list of sent and received MPI messages. You don't yet know which send belongs to which receive. Determining this relationship is called message matching. I intend to explain how it works in a future blog post.

Once this is done, you have one list of messages with correct durations, that can finally be used in your tool.

Happy Coding!

P.S.: I used OTF2 version 1.5.1

OTF2 Knowledge: How to Map Local MPI Ranks to OTF2 Locations

2015-05-29T08:18:00.000+02:00

OTF2 (part of the Score-P infrastructure) has a terrible reader interface. This post attempts to explain one little part of boilerplate you need to write in order to process MPI records.

For example, to process an MPI_Send record you use a handler with the following signature:

    OTF2_CallbackCode handleMpiSend(OTF2_LocationRef sender,
    OTF2_TimeStamp time, void* userData,
        OTF2_AttributeList* a, uint32_t receiver,
        OTF2_CommRef com, uint32_t tag, uint64_t length);

Notice that the data type of the sender and receiver processes are not the same. This is because the event is recorded as is during tracing, without trying to match MPI-specific, communicator-dependent identifiers to communicator-independent OTF2 ones. Since Score-P/OTF2 does not have a post-processing step, the connection between these identifiers and the OTF2 locations is never established. It is left to the programmer to apply this mapping while reading the trace.

Required Handlers

    OTF2_CallbackCode handleOtf2DefGroup(void *userData,
        OTF2_GroupRef group, OTF2_StringRef name,
    OTF2_GroupType groupType, OTF2_Paradigm paradigm,
    OTF2_GroupFlag groupFlags, uint32_t memberCount,
    const uint64_t* members);

    OTF2_CallbackCode handleOtf2DefCommunicator(void *userData,
        OTF2_CommRef com, OTF2_StringRef name,
        OTF2_GroupRef group, OTF2_CommRef parent);

Data Structures

map<OTF2_CommRef, OTF2_GroupRef> communicatorToGroup;

map<OTF2_GroupRef, map<uint32_t /*local rank*/,

uint64_t /*rank in comm world*/>> localRankToGlobalRank;

OTF2_GroupRef mpiLocationGroup;

The Algorithm

DefGroup

    localRankToGlobalRank[group] = {};
    for (uint32_t i = 0; i < memberCount; i += 1) {
        localRankToGlobalRank[group][i] = members[i];
    }

    if (type == OTF2_GROUP_TYPE_COMM_LOCATIONS &&
            paradigm == OTF2_PARADIGM_MPI) {
        mpiLocationGroup = group;
    }

DefCommunicator

communicatorToGroup[com] = group

Mapping Local MPI Ranks to OTF2 Locations

Input: local MPI rank rank, MPI communicator of this rank com

Output: OTF2 location

Procedure: First, map the local MPI rank to the rank in comm world. Next map this rank to the OTF2 location using the magic group mpiLocationGroup:

localRankToGlobalRank[mpiLocationGroup[localRankToGlobalRank[
communicatorToGroup[com]][rank]]

Happy Coding!

ps: I used OTF2 version 1.5.1

Automatically flushing QTextStream

2015-05-09T22:16:00.000+02:00

QTextStream is way more convenient for printing to console than C++ streams are. Therefore, I usually replace cerr and cout by qerr and qout in projects using Qt:

Header file:
     extern QTextStream qerr;
extern QTextStream qout;

Source file:
     QTextStream qerr(stderr, QIODevice::WriteOnly | QIODevice::Text);
     QTextStream qout(stdout, QIODevice::WriteOnly | QIODevice::Text);

Unfortunately, QTextStream buffers and is not able to automatically flush. This means it may print a message later than expected or not at all if the program crashes. This is obviously undesirable for a cerr replacement.

One way to work around this is by intercepting all stream operator calls and adding a call to flush():

Header file:
class AutoFlushingQTextStream : public QTextStream {
public:
AutoFlushingQTextStream(FILE* f, QIODevice::OpenMode o)
: QTextStream(f, o) {}

template<typename T>
AutoFlushingQTextStream& operator<<(T&& s) {
*((QTextStream*) this) << std::forward<T>(s);
flush();
return *this;
}
};

extern AutoFlushingQTextStream qerr;
extern QTextStream qout;

Source file:
AutoFlushingQTextStream qerr(stderr, QIODevice::WriteOnly | QIODevice::Text);
QTextStream qout(stdout, QIODevice::WriteOnly | QIODevice::Text);

Any Tips for improving the code are very much appreciated.

Minor Thesis - Development and Analysis of Barrier Protocols

2015-02-04T15:35:00.000+01:00

A year ago, I defended my minor thesis (equivalently bachelor thesis or Großer Beleg) (PDF download). The repository, including smaller course work on similar topics from before, is on GitHub: https://github.com/hydroo/barrier.

Thesis

1/2 Intro/Background

Cache coherency and atomic operations don't scale
Probabilistic synchronization is a good idea
Survey of currently used barrier protocols and of general ways to implement barriers
Detailed look at MPI 3's remote memory access API for implementing less deterministic barrier/synchronization protocols

3 Three Novel Barrier Protocols

MGB Barrier: Inspired by Nicholas Mc Guire

B1 Barrier: The opposite of the Central Counter Barrier
B2 Barrier: Variant of the B1 Barrier that computes more and communicates less

4 Evaluation

Central Counter Barrier vs B1 Barrier
Dissemination Barrier vs B2 Barrier
Some high-level observations about the four protocols
Verify correctness of the protocols via model checking
Compare the speed and energy consumption of the four barriers using quantitative model checking

Personal Comment

Sadly, I had to prioritize model checking over measurement-based analysis.
Therefore, I have little confidence in the quantitative model checking results. Aside from that, I quite like the thesis, and think there are valuable ideas in there. Especially:

The so-called progress problem that most barriers have. It is explained in Section 4.1.
The B1 and B2 Barrier can be faster than every other barrier protocol. Depending on the computation speed and communication latency different variants of them should be used. They avoid the progress problem. They have their weaknesses, too. More analysis is required to definitely state if the additional resource requirements outweigh the speed advantage.
The so-called barrier building blocks (Section 2.3)

Repository

Abstract

This is a semi-organized repository of ~1 year of university work. During this time I:

Surveyed existing barrier protocols
Surveyed actual implementations
Surveyed what low-level tools are available to develop synchronization primitives, and in particular barriers
Invented new protocols
Analyzed/modelchecked/benchmarked them
In-depth look at MPI 3's remote memory access API and its suitability for implementing (less deterministic) barriers and other synchronization protocols

License

Everything is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 license. If this licensing scheme is a problem for your intentions, please contact me. I'm open to suggestions.

Why upload the whole thing?

Because there are few reasons not to
I would oftentimes be very happy, if I could dig through the source code of someone's research
For people interested in researching barrier protocols and synchronization primitives in general
For the useful code snippets contained
As a portfolio entry so potential employers can see what I did and how I did it

New PGP and OTR Keys

2015-01-30T16:04:00.000+01:00

PGP key id: e94b3df55b06c8e4
OTR fingerprint: 1d070cab 116463a6 0f94b424 075b6c83 abb4a140

When in doubt, contact me some more reliable way, since this blog isn't a secure way of distributing this info.

Mouse cursor centered zooming in QGraphicsView

2014-04-11T16:19:00.000+02:00

Zooming into and out of a QGraphicsView so that the position under the mouse cursor remains fixed is easy, right? Sadly, it isn't. I leave my best working approach here for you to enjoy and hopefully save you some time.

Before coming up with this, I browsed the interwebs and tried a few other things. I always ended up with not-working code or a slight drift while repeatedly zooming.

void GraphicsView::wheelEvent(QWheelEvent* e) {
if ((e->modifiers()&Qt::ControlModifier) == Qt::ControlModifier
&& e->angleDelta().x() == 0) {

QPoint pos = e->pos();
QPointF posf = this->mapToScene(pos);

double by;
double angle = e->angleDelta().y();

if (angle > 0) { by = 1 + ( angle / 360 * 0.1); }
else if (angle < 0) { by = 1 - (-angle / 360 * 0.1); }
else { by = 1; }

this->scale(by, by);

double w = this->viewport()->width();
double h = this->viewport()->height();

double wf = this->mapToScene(QPoint(w-1, 0)).x()
- this->mapToScene(QPoint(0,0)).x();
double hf = this->mapToScene(QPoint(0, h-1)).y()
- this->mapToScene(QPoint(0,0)).y();

double lf = posf.x() - pos.x() * wf / w;
double tf = posf.y() - pos.y() * hf / h;

/* try to set viewport properly */
this->ensureVisible(lf, tf, wf, hf, 0, 0);

QPointF newPos = this->mapToScene(pos);

/* readjust according to the still remaining offset/drift
* I don't know how to do this any other way */
this->ensureVisible(QRectF(QPointF(lf, tf) - newPos + posf,
QSizeF(wf, hf)), 0, 0);

e->accept();
}

if ((e->modifiers()&Qt::ControlModifier) != Qt::ControlModifier) {
/* no scrolling while control is held */
QGraphicsView::wheelEvent(e);
}

}

Any Tipps to make this shorter/better are very much appreciated.

Unterhaltung

2014-02-04T00:00:00.000+01:00

Als mir erklärt wurde dass Spiele der Steam wishlist hinzuzufügen bedeutet dass man eine E-Mail bekommt wenn jenes Spiel im Angebot ist, habe ich das direkt genutzt um (hoffentlich) gute Spiele drauf zu packen damit ich die für wenig Geld mal ausprobieren kann.

Ein sehr schönes Spiel was ich so erworben hab' ist The Stanley Parable. Vielleicht nicht unbedingt 12€ Wert, da ziemlich kurz, aber auf jeden Fall ein sehr "interessantes" und unterhaltsames Werk.

Weiterhin bin ich auch mal durch Mass Effect 1&2 durchgerannt. Für 10€ ein guter Deal. Kurz gesagt ist Mass Effect ist ein stark gehypter 3rd person shooter der viel Wert auf Story und Auswahlmöglichkeiten legt.
Nachdem ich durch beide Spiele in weniger als 40h durch bin, war ich ein wenig enttäuscht, weil die Story nicht super ist (Für Neuere-Videospiel-Verhältnisse aber trotzdem eine der besten) und die übertriebene Dramatik der letzten Mission von Mass Effect 2 mich abgeschreckt hat.
Die letzte Mission war definitiv cool und packend, aber irgendwie hat mir das hinarbeiten auf den Klimax gefehlt. Bis zu dem Punkt war alles recht eintönig. Crew-Mitglieder kennenlernen. Loyalität mit einer einzigen Mission je Mitglied erarbeiten. Upgrades kaufen. Leveln. Usw.
Nach ein bisschen Recherche und Reden mit Freunden musste ich feststellen, dass ich die Auswahlmöglichkeiten und die verschiedenen Versionen der Story unterschätzt habe (z.B. hier das Video der schlimmsten Version der letzten Mission in ME2). Vielleicht spiele ich es demnächst nochmal durch, schlage dabei einen anderen Pfad ein und lass' mir etwas mehr Zeit dabei.
Glänzen können beide Spiele durch gute Charaktere, eine gute Steuerung, eine schöne Inszenierung, super Synchronsprecher und einen absolut genialen Soundtrack. Fazit: Absolut spielenswert.

Teil 3 liegt auch schon bereit. Man will ja wissen wie's weitergeht. Nur habe ich zur Zeit kein DVD-Laufwerk und EA's Steam-Konkurrenz namens Origin werde ich mir hoffentlich nie installieren. Einige EA-Titel wie z.B. Battlefield 4 und Mass Effect 3 sind nicht auf Steam verfügbar sondern nur auf Origin.

Vorgestern ist dann noch Mirror's Edge zur Sammlung, für 2,50€, dazu gekommen. Mirror's Edge ist ein first person shooter mit wenig shooting und setzt den Fokus auf Parkour als Spielemechanik. Nach den ersten paar Missionen muss ich sagen: fetzt. Mal sehen wie's weitergeht.

Zum Abschluss habe ich für alle die, wie ich, das Gefühl haben schon jeden neueren Sci-Fi-Film gesehen zu haben noch eine kleine Empfehlung: The Abyss. Unbedingt die ungeschnittene / non-Hollywood Variante ansehen.

Qt 5.2 + Android Error #1

2014-01-21T16:37:00.000+01:00

If you make a new project and try to debug/run you might get the following error.

Unable to start 'org.qtproject.example._8paint'. Debugging has failed

Starting remote process. Unable to start 'org.qtproject.example._8paint'

Notice the extra underscore in front of the program name. Apparently, your program's name is not allowed to begin with a number or an underscore.

edit: This error should be resolved in Qt 5.3

Lang lebe das Blog

2014-01-17T11:59:00.001+01:00

Der letzte Blog-Post liegt drei Monate zurück. Ups. Ich war ziemlich gestresst vom Beleg schreiben.

Mittlerweile ist der Beleg abgegeben, verteidigt und benotet. Alles lief besser als erwartet. Bin sehr zufrieden. Super!

Jetzt, da ich wieder in Wien bin und den Kopf einigermaßen frei habe, werde ich mal die Dinge in Angriff nehmen für die ich keine Nerven hatte. Meinen CV neu machen und auf github laden, Bürokram machen, zum Arzt gehen wegen meiner immernoch anhaltenden Fußverletzung, den Ungelesene-Bücher-Stapel verkleinern, bloggen und generell eigene Interessen verfolgen.

Bin erst ein paar Tage wieder in Österreich. Daher gibt es nichts zu berichten außer dass ich mal ein österreichisches Nationalgetränk ausprobiert habe. Hat, laut Aufdruck, sogar viele Vitamine! Schmeckt aber wie Zuckerwasser ohne definierbaren Geschmack. Nicht so toll.

Im Steam-Weihnachts-Sale habe ich mir Mass Effect 1 und 2, geholt und bisher die 1 durchgespielt und die 2 angefangen. Ziemlich ziemlich gut. Viel Story. Gute Story. Gute Dialoge. Gutes Gameplay. Keine auffälligen/nervigen Macken, abgesehen davon dass jeder Abschnitt komplett schlauchförmig ist. Fetzt.

EvE Online habe ich in letzter Zeit weniger gespielt. Sehenswert ist das folgende Video. Rooks and Kings stellt den remote starburst vor. Capital ships beschleunigen dabei auf 10,000 m/s und spielen Billard!

Vom 5.1. bis 11.1 fand der alljährliche Speedrun-Marathon Awesome Games Done Quick in Washington DC statt. Insgesamt wurde rund eine Million Dollar für die Prevent Cancer Foundation gesammelt. Alle Runs kann man hier finden. Ein paar ausgewählte runs:

Mike Tyson's Punchout!! (Ja, mit Ausrufezeichen im Titel), von Sinister1, mit verbundenen Augen. Genial!

Direkt danach kam noch die Steigerung (was ich nicht für möglich gehalten habe bei dem letzten run): Super Punch-Out!!, von Zallard1, mit verbundenen Augen. Spoiler: Er schafft es als erster überhaupt es durchzuspielen!

Minecraft speedrun! Mit festem seed, FYI. Ich wusste gar nicht, dass man es überhaupt rennen kann. Aber der junge Mann schafft es in unter 30 Minuten den Drachen zu besiegen. link

Von Ghost in the Shell gibt es eine neue übrigens eine neue Prequel-Reihe, die sich damit beschäftigt wie der Major zu Section 9 kommt und sein Team zusammenstellt. Bisher sind zwei Filme veröffentlicht worden. link

Die neue Anime-Season ist armselig, genau wie die Letzte. Viel Schul-Anime, Slapstick-Zeug und schlechte Dialoge. Schade.

Isaac Assimov's sieben Foundation Series-Bücher habe ich fast fertig gelesen. Der Mann hat's schwer drauf. Trotzdem habe ich das Gefühl dass die jungeren Bücher nicht so gut sind wie die ersten paar. Dennoch sehr lesenswert, genau wie die Robot-Geschichten.

Das war's erstmal. Bis demnächst!

Harmless Half-Knowledge

The Next Step for AI - Full Personal Interaction Capture

Load Local Vimrc Automatically From Parent Directories

Dell XPS 7590 SSD Benchmark

Diplom Thesis: Automatic Profile-Based Characterisation of Performance Traces for Highly Parallel Applications

Costa Rica - Wildlife Refuge Laguna Urpiano

Retirement Calculator

Great Smoky Mountains / Cades Cove Hike

Dark Souls 1 Photo Tour

In a Galaxy Far Far Away

Intel SSD 600p (1TB) Luks+Ext4 Encrypted Disk Performance

LUKS+Ext4

Ext4 Only

Evaluation

Race Report: 19ter Lauf in den Frühling

Race Report: 19ter Oberelbe-Marathon

Race Report: 26ter Citylauf Dresden

Thank you From Software

Error with Scripting Vim in Python - Ubuntu Xenial

OTF2 Knowledge: How to Match MPI Messages

OTF2 Knowledge: How to Process Non-Blocking MPI Messages

Explaining the Involved Records

Data Structures

Send/Receive

Non-blocking

The Algorithm

Send

Isend

IsendComplete

Receive

IreceiveRequest

Ireceive

RequestCancelled

Wrap Up

OTF2 Knowledge: How to Map Local MPI Ranks to OTF2 Locations

Required Handlers

Data Structures

The Algorithm

DefGroup

DefCommunicator

Mapping Local MPI Ranks to OTF2 Locations

Automatically flushing QTextStream

Minor Thesis - Development and Analysis of Barrier Protocols

Thesis

Contents

Personal Comment

Repository

Abstract

License

Why upload the whole thing?

New PGP and OTR Keys

Mouse cursor centered zooming in QGraphicsView

Unterhaltung

Qt 5.2 + Android Error #1

Lang lebe das Blog