Plain Old Stan

Testing Traefik's dyn.toml jinja template generation with python.

2024-11-23T13:46:00.001+01:00

Thought I'd share the simple python code to dry run the jinja template with variables. I needed to do it for the Traefik proxy dyn.toml. Especially with their change of using multiple Host and PathPrefix conditions, instead of having multiple parameters inside these.

Here is the code:

from jinja2 import Environment, select_autoescape, FileSystemLoader

env = Environment(loader=FileSystemLoader(
searchpath='./'))

context = {
    'fqdn': 'xx1.blocoware.com',
    'http_services': [
{
    'name': 'adiona',
    'priority': '2',
    'path_prefix': '`/prefix1/`, `/prefix2/`',
    'all_fqdn_aliases': 'zz1.blocoware.com,xx1.blocoware.com,yy1.blocoware.com',
    'ip_port': '127.0.0.1:111'
},
{
    'name': 'beacon',
    'priority': '4',
    'path_prefix': '`/prefix2/`',
    'all_fqdn_aliases': 'zz1.blocoware.com,xx1.blocoware.com,yy1.blocoware.com',
    'ip_port': '127.0.0.1:333'
}
]}

template = env.get_template('dyn.toml.j2')

body = template.render(context)

print(body)

And fragment of the dyn.toml.j2 to show the usage of the context:

 [http.routers]
{% for service in http_services %}
  {% set domains = all_fqdn_aliases.split(',') %}
  {% set prefixes = service.path_prefix.split(',') %}
  [http.routers.{{ service.name }}]
    rule = "({% for domain in domains %}Host(`{{ domain }}`){% if not loop.last %} || {% endif %}{% endfor %}) && ({% for prefix in prefixes %}PathPrefix({{ prefix }}){% if not loop.last %} || {% endif %}{% endfor %})"
    entrypoints = ["main", "main-secure"]
    ...

Rendered fragment:

  [http.routers.adiona]
    rule = "(Host(`zz1.blocoware.com`) || Host(`xx1.blocoware.com`) || Host(`yy1.blocoware.com`)) && (PathPrefix(`/prefix1/`) || PathPrefix( `/prefix2/`))"
    entrypoints = ["main", "main-secure"]
    ...

These are my first lines of python code ever, so I'm letting you to cope with the python environment setup.

Somehow I confirmed to vscode to create this environment in some folder and I had to copy dyn.toml.j2 there, otherwise it could not find it, regardless the searchpath. Run button is fortunately just above the python window :). You'll need to install python extension, but vscode offers that once you create your first python file, cool.

Hope it might help someone.

My regards to all jinjas!

Resources:

Jinja: https://jinja.palletsprojects.com/en/stable/

Traefik Host and PathPrefix:

https://doc.traefik.io/traefik/routing/routers/#host-and-hostregexp

https://doc.traefik.io/traefik/routing/routers/#path-pathprefix-and-pathregexp

Elastic 7.1 > 8.16. Journalbeat > Filebeat. Migration notes.

2024-11-16T20:08:00.005+01:00

Journalbeat is removed in 7.16

https://github.com/elastic/observability-docs/issues/1173

Use filebeat's new input type of journald:

https://www.elastic.co/guide/en/beats/filebeat/master/filebeat-input-journald.html

Add input:

filebeat.inputs:
...
- type: journald
  id: everything

To avoid indices like:

filebeat-%{[fields][source]}-2024.11.16

Set fields, source in journald input type, e.g.:

filebeat.inputs:
...
- type: journald
  id: everything
  fields:
    source: journal

That will lead to index names as:

filebeat-journal-2024.11.16

I was trying with the index name configuration as described in docs:

https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-journald.html#_index_13

But I should have been doing something wrong, as I was only getting indices like filebeat-%{[fields][source]}-2024.11.16.

Failed Proxmox 7 -> 8 upgrade. I forgot to substitute bullseye with bookworm in one of the repository lists!

2024-11-13T15:48:00.002+01:00

They literally say in Proxmox wiki for 7 to 8 upgrade:

Ensure that there are no remaining Debian Bullseye specific repositories left, if you can use the # symbol at the start of the respective line to comment these repositories out. Check all files in the /etc/apt/sources.list.d/pve-enterprise.list and /etc/apt/sources.list and see Package_Repositories for the correct Proxmox VE 8 / Debian Bookworm repositories.

And I managed to forget the one in the /etc/apt/sources.list.d/pve-enterprise.list. I should not have had it there in the first place, at that time I didn't know Proxmox at all.

So after following the upgrade steps and having no errors after apt dist-upgrade I faced no Web interface being available as the first sign of my failure.

journalctl -u pveproxy showed:

Nov 13 14:24:08 pve62 pveproxy[314931]: ipcc_send_rec[3] failed: Connection refused

systemctl status pve-cluster showed:

○ pve-cluster.service
     Loaded: masked (Reason: Unit pve-cluster.service is masked.)
     Active: inactive (dead)

pveversion:

-bash: pveversion: command not found

Running apt update showed me that I have bullseye still somewhere in the sources.

Where?

List the repositories:

grep "^[^#]" /etc/apt/sources.list /etc/apt/sources.list.d/*

/etc/apt/sources.list:deb http://ftp.cz.debian.org/debian bookworm main contrib
/etc/apt/sources.list:deb http://ftp.cz.debian.org/debian bookworm-updates main contrib
/etc/apt/sources.list:deb http://security.debian.org bookworm-security main contrib
/etc/apt/sources.list.d/pve-enterprise.list:deb http://download.proxmox.com/debian/pve bullseye pve-no-subscription

Fix:

vim /etc/apt/sources.list.d/pve-enterprise.list

What now? Let see how to put broken proxmox installation back on top of debian:

https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_12_Bookworm

Running:

wget https://enterprise.proxmox.com/debian/proxmox-release-bookworm.gpg -O /etc/apt/trusted.gpg.d/proxmox-release-bookworm.gpg

Now:

apt install proxmox-ve

Fails with error:

E: Failure running script /usr/share/proxmox-ve/pve-apt-hook

Here:

https://forum.proxmox.com/threads/upgrade-failed-pve-apt-hook-not-found.107950/post-464053

Quoting:

I did the following and that seems to have solved the problem.

mkdir -p /usr/share/proxmox-ve
touch /usr/share/proxmox-ve/pve-apt-hook
chmod +x /usr/share/proxmox-ve/pve-apt-hook

After that:

apt update && apt full-upgrade

apt dist-upgrade

And the last one:

apt install proxmox-ve

Web inteface is back, no errors so far. All containers are running as they should.

ML310e Gen 8 v2 - Front IO pinout. Using ML310e case as a MicroATX housing.

2024-09-21T18:29:00.005+02:00

Recycling now my very old ML310e and want to reuse it for the X570D4U from AsRock.

Here is a hole I had to cut in ML310 the case, lol:

And as I was not able to find any pinout diagram for the front-io panel:

Here is mine (don't trust me though and always re-check by yourself!):

Sending my worldwide regards to all server recyclers :)!

Example of a PostgreSQL function to convert json string literal to a table.

2021-12-11T21:51:00.001+01:00

Was not able to google one up readily, so created this sample:

Hope it might help someone!

TIL: Proxmox "pct create" -rootfs parameter and disk size.

2021-10-22T14:10:00.001+02:00

Got confused by that -rootfs parameter of proxmox when creating a new container with "pct create", so documenting it here.

-rootfs local:2

translates into (when creating container with id 1062, /etc/pve/lxc/1062.conf):

rootfs: local:1062/vm-1062-disk-0.raw,size=2G

so local:2 then stands for the "local" volume and 2GB as a disk size.

Using size=2G sub-parameter led to the size of 1062G, interestingly. Not sure if this is a bug in proxmox (v7.0) or what.

Ubuntu 18.04 server and ESXI on Intel NUC Hades Canyon.

2019-12-07T22:58:00.001+01:00

This post is just to confirm that dual boot Ubuntu 18.04.3 server and ESXI 6.7 U3 on Intel NUC Hades Canyon is possible and to describe some pitfalls/obstacles I had to jump through.

My configuration for the Intel NUC Hades Canyon is Kingston dual ranked 2x16GB + 2xSamsung EVO 670 500GB.

I first thought I'd do it with RAID in BIOS, but both ESXI and Ubuntu server just ignored this option anyway and second, I read that actually it is a good idea to avoid this BIOS RAID option in order to be able to read/use disks after the CPU/board failure.

#1 Installing ubuntu 18.04.3 server on Intel NUC Hades Canyon

I should tell I was a long time user of Digital Ocean, Host Europe and so on - preinstalled Ubuntu server was it for me. Somehow this time I bumped into "live" version of Ubuntu server and started to install it with the outcome of installation not seeing the disks to install Ubuntu, "Unfortunately Probing for devices to install to failed.":

https://bugs.launchpad.net/subiquity/+bug/1841333

This "problem" was the reason for the part #2 of this post - installing ESXI. I only got to the article above and understanding that I picked the wrong Ubuntu version after I have ESXI successfully installed as a boot option for disk 1.

Here is the image for Ubuntu 18.04.3 "not live" server, just for a reference:

http://cdimage.ubuntu.com/releases/18.04.3/release/ubuntu-18.04.3-server-amd64.iso

I also ran into Ubuntu 18.04 server installation getting stuck at 66% while running'update-grub':

https://unix.stackexchange.com/questions/511289/ubuntu-18-04-server-installation-gets-stuck-at-66-while-runningupdate-grub

After killing the offending process, to exit busybox I used:

```
exit
```

and then probably Ctrl+F1

#2 Installing VMWare ESXI Hypervisor

My main problem with ESXI installation was creating a bootable ESXI image on my mac. After a bit of research I was lucky enough to find Unetbootin:

https://unetbootin.github.io

Other than that, ESXI was a breeze to install and use. Btw, I used the "live" version of Ubuntu 18.04.3 server as ESXI ubuntu iso image and it worked just alright, I really liked the ssh from github option when installing.
--------

This concludes my notes on dual boot Ubuntu 18.04.3 and ESXI on Intel NUC Hades Canyon. Machine performance is outstanding, no heat, no noise. Performance of AWS large instance to paid for itself within 45 days from what I read.

Next steps are to study on MAAS, LXD, Juju for the Kubernetes deployment as a target.

Also looking forward the next generation of Intel NUC Canyon, the Ghost one!

xCode 10 Swift 4.2 slow autocomplete and compile times. BAD.

2019-02-20T14:47:00.001+01:00

At some point I started to have dismal swift autocomplete delays in xCode 10 and slower compile times. Having googled this and that and optimizing what I could I also found into these type of warnings:

My first guess was that longer tuples are causing the problem. I changed this part to use structs, no joy.

One more part in the code gave me another hint:

Looked to me as if just adding up literals via "+" would be tricky for swift compiler's type checking.

Changing everything above to String(format:) resolved the type checking speed.

If you ask on using string interpolation with \() - it didn't solve the problem. Plus I don't like these messy strings when they get longer.

I have no good words for the guys that work on Swift programming language and its design/tools. I don't think this part and types of optimization should really ever bother us, regular devs.

Bad, bad, bad.... I wish I could have loved Swift more. I love the language even if I don't think it is superior that much to Objective-C. Moving to it from Objective C with all new and updated stuff, but this is simply bad.

P.S. You can find a really good digest of Swift compile time optimizations here:

https://github.com/fastred/Optimizing-Swift-Build-Times

TIL - Elixir Jason.Encoder - handling optional fields

2019-01-01T14:23:00.001+01:00

At some point along my learning road I had a location struct with the following fields:

@derive Jason.Encoder

defstruct latitude: -999, longitude: -999, speed: -1.0, altitude: -1.0, bearing: -1.0

I added one more field:

defstruct latitude: -999, longitude: -999, speed: -1.0, altitude: -1.0, bearing: -1.0, timestamp: 0.0

And my tests for encoding the struct values after reading these older location records from a file started to fail.

Turns out this @derive part is generating the code similar to this:

defimpl Jason.Encoder, for: [Location] do

def encode(%{altitude: _, bearing: _, latitude: _, longitude: _, speed: _, timestamp: _} = value, opts) do

Jason.Encode.map(Map.take(value, [:altitude, :bearing, :latitude, :longitude, :speed, :timestamp]), opts)

end

end

ref: https://hexdocs.pm/jason/Jason.Encoder.html#content

So older records can't be matched by this automatically generated definition.

Considering that as a part of "support older data with a fallback value for the new field" exercise and based on Jason.Encoder documentation:

1. I removed the @derive Jason.Encoder line to avoid the automatic protocol implementation generation.

2. Provided own implementation that supports the old data stored:

require Jason

defimpl Jason.Encoder, for: [Location] do

# Matches the default implementation of @derive Jason.Encoder

def encode(%{altitude: _, bearing: _, latitude: _, longitude: _, speed: _, timestamp: _} = value, opts) do

Jason.Encode.map(Map.take(value, [:altitude, :bearing, :latitude, :longitude, :speed, :timestamp]), opts)

end

# Support for older data

def encode(%{altitude: _, bearing: _, latitude: _, longitude: _, speed: _} = value, opts) do

Jason.Encode.map(Map.take(value, [:altitude, :bearing, :latitude, :longitude, :speed]), opts)

end

end

This it for my Today I Learnt.

References:

https://hexdocs.pm/jason/Jason.Encoder.html#content

https://github.com/michalmuskala/jason

And as this post started with adding a timestamp field, here you can read on timestamps in Elixir. From the Jason.Encoder author:
https://michal.muskala.eu/2017/02/02/unix-timestamps-in-elixir-1-4.html

Copying comments in Google spreadsheet into a column

2017-08-14T14:16:00.002+02:00

Will copy every comment on a cell in a column into a cell next to the source column (the one with the comments):

function commentsToColumn() {

// find selection (the source comments)

var workbook = SpreadsheetApp.getActiveSpreadsheet();

var sheet = workbook.getActiveSheet();

var source = sheet.getActiveSelection();

// determine target (the column to the left of the selection)

var target = source.offset(0, +1);

// copy comments

comments = source.getComments();

target.setValues(comments);

}

If you need more info for usage, please ask in comments :).

iOS 11 beta 1,2 - NSString sizeWithAttributes - breaking change and potential bug

2017-07-02T10:41:00.000+02:00

Edge behavior of [NSString sizeWithAttributes] seems to be changed in iOS 11 beta 1 and 2. When asked for the

float size around 0.0;

NSString *testString = @"Some test string";

NSString *fontName = @".SFUIText-Semibold";

UIFont *targetFont = [UIFont fontWithName:fontName size:size];

CGSize stringUISize = [testString sizeWithAttributes:@

{

NSFontAttributeName: targetFont

}];

You may get very different from iOS 7- 10 values. Here is the test:

While running it on iOS 7-10, everything converges to zero string UI width:

2017-07-02 10:11:45.996 speedometer[23301:7084973] attempted font size: 1, calculated string width: 12.157471, delta: -12

2017-07-02 10:11:45.997 speedometer[23301:7084973] attempted font size: 0, calculated string width: 4.052490, delta: -4

2017-07-02 10:11:46.005 speedometer[23301:7084973] attempted font size: 0, calculated string width: 0.000000, delta: 0

while on iOS 11 beta 1 and 2 the same test case is "stuck" on value 97 something on iPhone 6 Plus:

2017-07-02 08:29:07.157511+0200 speedometer[36517:257340] attempted font size: 4, calculated string size: 36.437256, delta: -36

2017-07-02 08:29:07.158646+0200 speedometer[36517:257340] attempted font size: 3, calculated string size: 28.340088, delta: -28

2017-07-02 08:29:07.159723+0200 speedometer[36517:257340] attempted font size: 2, calculated string size: 20.242920, delta: -20

2017-07-02 08:29:07.160738+0200 speedometer[36517:257340] attempted font size: 1, calculated string size: 12.145752, delta: -12

2017-07-02 08:29:07.161953+0200 speedometer[36517:257340] attempted font size: 0, calculated string size: 4.048584, delta: -4

2017-07-02 08:29:07.166740+0200 speedometer[36517:257340] attempted font size: 0, calculated string size: 97.166016, delta: -97

2017-07-02 08:29:07.167756+0200 speedometer[36517:257340] attempted font size: -1, calculated string size: 97.166016, delta: -97

2017-07-02 08:29:07.168736+0200 speedometer[36517:257340] attempted font size: -2, calculated string size: 97.166016, delta: -97

Looks to me as a broken "contract" and invariant. One would really expect the consistent behavior as it was since iOS7 and having the width of the string calculated as zero for zero font size and zero it should be downwards? Also string width jumping from about approaching zero values to 97.166016 when value got a bit on below zero looks inconsistent. Probably a missing unit test :)!

For me it looks like a bug and I'm reporting it to Apple now. Hopefully it will be fixed to the same consistent behavior as it used to be on iOS 7 - 10.

Defensive coding is like an alloc/release for me, I try to make it right and upfront, but doing static analysis later I found myself being "mentally away" way too often.

Yours and wishing you happy coding!

Stan.

UINavigationController - ask for confirmation on Back

2017-04-11T17:01:00.000+02:00

Didn't expect this to be such a problem, but once you tap into the UINavigationController things become hairy.

Requirement:

When user leaves a screen by tapping on a "back" in navigation bar and there are changed data in the screen I should ask for a confirmation and keep user at the current UIViewController if she decided to continue editing the data.

Solution.

You may run into this: http://stackoverflow.com/questions/1214965/setting-action-for-back-button-in-navigation-controller/19132881#19132881 (particularly this: https://github.com/onegray/UIViewController-BackButtonHandler).

Once, I was trying to solve the keyboard accessory to be shown each time for each UITextField on shouldBeginEditing by writing a category for a UITextField. And here is something I learned in a hard way:

When you plan or see any category re-writing the existing framework method, STOP! Simple as this and go read on what can turn wrong with this.

The solution mentioned above use this:

@implementation UINavigationController (ShouldPopOnBackButton)

- (BOOL)navigationBar:(UINavigationBar *)navigationBar shouldPopItem:(UINavigationItem *)item {

 if([self.viewControllers count] < [navigationBar.items count]) {

No go for me, no need to study, excuse for being abrupt :). But one part of code from this solution turned actually to be useful.

One of the comments on another post: http://stackoverflow.com/questions/20327165/popviewcontroller-strange-behaviour got me here: http://blog.macca.tech/2013/11/ios-prevent-back-button-navigating-to.html

And was not I lucky? It really makes sense, no private APIs, framework's UIViewController gets the chance to do its stuff always. What I wanted to improve though was that "safeDelegate" property and the way it is established. So I added a new method (into UISafeNavigationController.m):

-(id<UISafeNavigationDelegate>) popDelegate
{
    UIViewController *topController = [self topViewController];
    
    if ([topController conformsToProtocol:@protocol(UISafeNavigationDelegate)]) {
        return (id<UISafeNavigationDelegate>)topController;
    }
    
    return nil;
}

And then you can just substitute safeDelegate with popDelegate:

- (UIViewController *)popViewControllerAnimated:(BOOL)animated
{
    if (self.popDelegate && ![self.popDelegate navigationController:self
                                             shouldPopViewController:[self.viewControllers lastObject]
                                                                 pop:^{ [super popViewControllerAnimated:animated]; }])
    {
        if (self.navigationBar) {
            [self restoreViewsForNavigationBar:self.navigationBar];
        }
        return nil;
    }
    
    return [super popViewControllerAnimated:animated];
}

Also note lines 7-9 where I call a new method (borrowed from the first stackoverflow solution that I actually criticize :)):

-(void) restoreViewsForNavigationBar: (UINavigationBar *) navigationBar
{
    for(UIView *subview in [navigationBar subviews]) {
        if(0. < subview.alpha && subview.alpha < 1.) {
            [UIView animateWithDuration:.25 animations:^{
                subview.alpha = 1.;
            }];
        }
    }
}

This is to avoid the back arrow in the navigation bar looking as disabled when answer from our controller to the shouldPop is NO.

Then protocol method in the related view controller may look like:

- (BOOL)navigationController:(UINavigationController *)navigationController
     shouldPopViewController:(UIViewController *)controller pop:(void(^)())pop
{
    if (!_item.id) {
        UIAlertController *alert = [UIAlertController alertControllerWithTitle:LSSTRING(@"Save the item?") message:LSSTRING(@"You are closing this screen by using Back button and have not saved the item.") preferredStyle:UIAlertControllerStyleAlert];
        
        [alert addAction:[UIAlertAction actionWithTitle:LSSTRING(@"Save and close") style:UIAlertActionStyleDefault handler:^(UIAlertAction *action) {
            [self done:nil];
        }]];
        
        [alert addAction:[UIAlertAction actionWithTitle:LSSTRING(@"Don't save and close") style:UIAlertActionStyleDestructive handler:^(UIAlertAction *action) {
            [self doCancellationCleanup];
            if (pop) {
                pop();
            }
            
        }]];
        
        [alert addAction:[UIAlertAction actionWithTitle:LSSTRING(@"Cancel") style:UIAlertActionStyleCancel handler:^(UIAlertAction *action) {
            
            
        }]];
        
        [alert show];
        return false;
    }
    
    return true;
}

Hope this can help someone! All the credit goes to Hong Kong Web Entrepreneur guy! Once again, here: http://blog.macca.tech/2013/11/ios-prevent-back-button-navigating-to.html

[UPDATE] In the end I had to rework the final part presented here - showing the confirmation as the way it is presented was not releasing the controller correctly and was not cleaning up the views as well. Quite bigger effort is required to get it right and it is not that generic in the end. But this is a good start! :).

Debugging in elixir. Old erlang version got into my way.

2017-04-09T13:24:00.000+02:00

Today, some dialyzer stuff, but mainly debugging with Elixir was my study subject and so first I ran into iex telling me "Could not find 'wxe_driver.so'" on running :debugger.start(). Then I read that debugger starts anyway, so I tried simple :int.ni(Module), but was greeted with "no beam found for Elixir.Module". Read more and it pointed to Erlang version being lower than 19. Really iex and elixir --version were telling its 18 that is being started. Using brew upgrade erlang or elixir, brew reinstall - same thing. A year ago I installed erlang from the official installer here (https://www.erlang-solutions.com/resources/download.html). Looked there and there is an even version manager for the Erlang on OSX now?! Cool:

This surely helped to have version of erlang as 19.3 for elixir now and all the debugger stuff worked as expected.

I'm looking more and more into the Sasa Juric blackjack example as I'm kind of getting a bit sad about the book I wanted to read as a practical guide to building elixir/OTP apps: https://pragprog.com/book/lhelph/functional-web-development-with-elixir-otp-and-phoenix
I'm kinda on Sasa's side of analysis how the functional apps should be built and the book mentioned is not leading in the right way I believe. I'm still at its start though, author may still refactor his approach later... :).

Sasa uses dialyzer in his example and I read before that it is a good idea to use it, so studied today on this and typespecs in Elixir as well: http://elixir-lang.org/getting-started/typespecs-and-behaviours.html. I also loved a TDD example on building roman numbers converter here: https://medium.com/@barruumrex/seeking-simple-satisfaction-2a098902ddff

Elixir - bind the match subject in case statement

2017-04-05T13:05:00.001+02:00

Before I forget again on how to bind the whole match subject in case statement in Elixir:

"state" as fn argument name and "state" in case surely are having the different scope, that's just I wanted to have the same name saying "state" :).

Reading now "Functional Web Development with Elixir, OTP, and Phoenix", so this is my version of authors to_string:

def to_string(coordinate) do

"(in_island:#{island(coordinate)}, guessed:#{guessed?(coordinate)})"

end

I thought I'd write it in a way that I would not call the wrapping functions on Agent for each interpolated field, plus would avoid changing to_string implementation with every struct change.

By the way, new cool link for learning Elixir:

https://elixirschool.com

Also, just in time for reading the book mentioned above comes an article from Saša Jurić:
To spawn, or not to spawn? Definitely going to validate the book suggested solution/architecture with what Saša Jurić is writing.

Done with learning for today, back to creating some added value to my customers!

xcode 8.3 CompileStoryboard Internal error. UICGColor encodeWithCoder. Please file a bug at bugreport.apple.com and attach ...

2017-03-30T18:12:00.002+02:00

Was dancing with my voodoo drum for a few minutes after upgrading to xcode 8.3 today. One of my storyboards that have existed for last 3 years suddenly started to cause the compile error:

CompileStoryboard bLocNote/taxi/EditTariffStoryboard.storyboard

2017-03-30 17:48:00.332 Interface Builder Cocoa Touch Tool[29650:5173036] *** Assertion failure in -[UICGColor encodeWithCoder:], /BuildRoot/Library/Caches/com.apple.xbs/Sources/UIKit_Sim/UIKit-3600.7.47/UIColor.m:1549

2017-03-30 17:48:08.466 ibtoold[29647:5172916] [MT] IBPlatformTool: Wrote failed marshalling request diagnostics to path: /var/folders/8j/x30frbx948l8djrb6ff7mljc0000gn/T/IB-agent-diagnostics_2017-03-30_17-48-08_415000

/* com.apple.ibtool.errors */

/Users/stanislavdvoychenko/Documents/code/speedo/bLocNote/taxi/EditTariffStoryboard.storyboard: error: Internal error. Please file a bug at bugreport.apple.com and attach "/var/folders/8j/x30frbx948l8djrb6ff7mljc0000gn/T/IB-agent-diagnostics_2017-03-30_17-48-08_415000".

Was compiling fine just an hour ago on xcode 8.2.1! Ok, it tells something on not liking the UIColor, and log it mentions further says colors should be RGB or White space. I went by "binary search" removed everything from the storyboard except the table view -> build still fails. Only value catching the eye was "Tracking" color for the section view saying showing a black stripe with no color name, I have many of these assigned in other places though, guess it is still somehow different in this storyboard, so I reselected a "black" color for it:

And it compiled! Huraaay and back to programming!

Inherited uiviewcontrollers references in one storyboard. This class is not key value coding-compliant for the key ...

2017-03-16T19:05:00.000+01:00

I often inherit my UIViewControllers to add functionality on top of already existing. All works fine, but today I ran into the weird problem. Setup like this:

Where rangeItemEditor view controller inherits from itemEditor view controller. All outlets connected in the parent view controller. On row tap I do performSegue and then this:

This class is not key value coding-compliant for the key ...

Complaining about one of the outlets. After standard troubleshooting, checking that .destinationViewController is right, bunging my head a bit against the wall and trying [navigationController push] which worked I looked more carefully at the Storyboard ID values for these references. Here is obviously a good one:

And the other one, inherited rangeItemEditor had it empty. Filling it in and running again proved the idea that missing Storyboard ID in Identity tab for a references storyboard is a bad thing. I was not clearing this out though, not sure how it happened to be empty.

Thought I'd share to make your troubleshooting of similar cases faster!

In the end I got rid of these references in the storyboard and used [navigationController push]. Why? I just really don't want these crashes when xcode UI Editor decides to remove something behind the scenes. I've witnessed this "open the file -> get the mess" already in xcode storyboards and I really better stay safer here with using old gold manual push. Staying away from the magic until it has predictable results :).

Working around WITH not being available in older versions of sqlite.

2017-02-20T17:45:00.001+01:00

For one of my iPhone apps I need to rename route points to match their order as they are inserted or deleted, I started with an easy WITH version:

[NSString stringWithFormat:@"WITH wcte (id, wname) AS (SELECT w.id, '%@ ' || (SELECT COUNT(*) + 1 FROM waypoint WHERE vY1 < w.vY1) as wname FROM waypoint w) UPDATE waypoint SET \"name\" = (SELECT wname FROM wcte WHERE id = waypoint.id) WHERE \"name\" LIKE '%@ %%' OR \"name\" is null OR \"name\" = ''", LSSTRING(@"Point"), LSSTRING(@"Point")]

And ran into the problem when testing on iOS8.1, obviously its version of sqlite didn't support WITH at that time.

So here is the workaround solution for older sqlite versions:

[NSString stringWithFormat:@"UPDATE waypoint SET \"name\" = (SELECT '%@ ' || (SELECT COUNT(*) + 1 FROM waypoint w WHERE w.vY1 < w1.vY1) FROM waypoint w1 WHERE w1.id = waypoint.id) WHERE \"name\" LIKE '%@ %%' OR \"name\" is null OR \"name\" = ''", LSSTRING(@"Point"), LSSTRING(@"Point")]

Not that nicely looking, but I'm still committed to support iOS8 for a few more months. If you are puzzled by that LSSTRING part - that's just my macro for the LocalizableString as I only want to rename these points that are named automatically and surely I want to name them in the localized manner. Punto it is in Spanish (I hope) :). vY1 is a cryptic name for the order column :).

Would not be publishing at all, but sqlite syntax sometime is surprising in what it can or can't do, so I thought I might save time to someone.

If you are into hiking, fishing, cycling or classic skiing here is the app link, it's free: https://itunes.apple.com/us/app/id1120906807

Elixir, Phoenix, React, Redux and ES6 - I'm a complete beginner again.

2016-12-09T15:02:00.002+01:00

I started with Erlang and now progressing with Elixir, Phoenix, React, Redux and ES6 stack. Was feeling a bit lost when seeing stuff like this:

var createStoreWithMiddleware = (0, _redux.applyMiddleware)(reduxRouterMiddleware, _reduxThunk2.default, loggerMiddleware)(_redux.createStore);

or:

const store = autoRehydrate()(createF8Store)(reducers);

Had to remind myself on a simple function chaining, like:

const p = (store)=>{console.log("I'm an almighty container, you passed me the store: " + store);};

const f = (a,b)=>{

console.log("Thanks for a and b: " + a + ", " + b + " it helped me to build a container with I return to you. Pass a store into it!");

return p;

};

f(3,4)("Store is just a name 'John'");

With the output of:

"Thanks for a and b: 3, 4 it helped me to build a container with I return to you. Pass a store into it!"
"I'm an almighty container, you passed me the state: Store is just a name 'John'"

Beginner mind work play games with me, tendency to get lost on trivial subjects is quite high :):):).

A list of resources I really appreciated so far:

React:
https://facebook.github.io/react/docs
https://egghead.io/courses/react-fundamentals
https://egghead.io/courses/react-flux-architecture-es6

Redux:
http://redux.js.org
https://egghead.io/lessons/javascript-redux-reducer-composition-with-combinereducers

ES6:
https://ponyfoo.com/articles/es6

ES6 and modules (oh, how could have I got so much behind?):
https://24ways.org/2014/javascript-modules-the-es6-way/
https://babeljs.io/learn-es2015/

ES6 Promises: http://www.datchley.name/es6-promises/

I also found this article being an awesome walkthrough on getting React/Redux/ES6 and more together in Facebook's makeitopen app:
http://makeitopen.com/tutorials/building-the-f8-app

Webpack:
http://ryanchristiani.com/introduction-to-webpack/

Elixir and Phoenix:
Pragmatic programmers - Elixir programming, Phoenix programming.
Phoenix and Trello clone: http://codeloveandboards.com/blog/2016/01/14/trello-tribute-with-phoenix-and-react-pt-1/

Me, a troll! My recent MS experience :)

2016-06-15T11:19:00.002+02:00

If I had to show one example for all for my recent Microsoft experience as a developer and user, here it is. The welcome page in VS 2015 opening the xml feed when you click on "More news":

I will not be telling here how I'm trying to keep my ASUS tablet in sync with time after upgrading to Windows 10. The above is enough declaration of "we are not for humans" and "we don't really care" :).

This is me, a troll in the MS land. I used to be a regular guy here. Sorry. I'm working hard on my escape plan now.

Stan.

P.S. To my problem with Windows 10 time. Given how different apps in windows fail differently based on that time sync problem and learning Erlang at the moment, it was quite up to point to read about Erlang time here: http://learnyousomeerlang.com/time, wishing you a good read! :).

Symbolicating bitcode crash logs in XCode

2016-05-26T16:11:00.002+02:00

As of Xcode version 7.3 there is still a problem with symbolicating the bitcode crash logs. I've attempted several solutions, here and here, but here is what I came to:

1. Crash logs provided are for the dSYMs that Apple also provides. Just go to the iTunes connect and download them:

Once you have them right click on the crash in xcode -> "Show in finder" and copy to some target directory - that's actually a crashpoint file you are going to get.

My target directory is now ~\Desktop\crashes and here is how it looks:

Where #1 are dSYMs as copied from the iTunes connect. #2 is the crashpoint saved from the crash organizer window in xCode. #4 is a crash log extracted from the crashpoint file (right click and "show package contents")

To get #3 - a symbolicatecrash app, execute this in the target directory (xcode 7.3):

cp /Applications/Xcode.app/Contents/SharedFrameworks/DVTFoundation.framework/Versions/A/Resources/symbolicatecrash symbolicatecrash

For different versions of xcode the location of symbolicatecrash will be different.

To get symbolicatecrash ready to operate execute:

export DEVELOPER_DIR='/Applications/Xcode.app/Contents/Developer'

Only one step is left! That's to get your crash log symbolicated:

./symbolicatecrash a.crash dSYMs/ > a.log

And the output file a.log is looking like:

Way better, is not it?

There are nuances that I would not touch here, like these dSYM files you downloaded are for different architectures and it would be the best to match your specific crash log to a corresponding dSYM. I'll leave this detail to you, my experience is that function names do match quite well, then it is just about the line of code information you'll be getting. In above picture, given crash and dSYM are matched well, you can see the line of code where the crash exactly happened. Otherwise you'd see +xyz there and that's not that helpful.

This is it. Please don't judge me strictly, I'm not an expert in any field, as I learnt over time :)! Knowing more than I do? Share your knowledge in the comments!

Yours,
Stan.

Erlang http response emulator

2016-04-27T13:59:00.002+02:00

Recently I had to implement a "transparent" proxy between Tibco and Blackberry. I thought I'd share the erlang module I wrote to emulate responses from BB as you can't really expect from BB server to give you error responses or timeouts so you can test your proxy part. Here it is:

-module(httpemulator).
-author("stanislavdvoychenko").

-export([client/0, server/0,start/0,accept/1,enter_loop/1,loop/1]).

client() ->
  {ok, Socket} =  gen_tcp:connect("localhost", 4001,[list, {packet, 0}]),
  ok = gen_tcp:send(Socket, "packet"),
  receive
    {tcp,Socket,String} ->
      io:format("Client received = ~p~n",[String]),
      io:format("Client result = ~p~n",[String]),
      gen_tcp:close(Socket)
  after 1000 ->
    exit
  end.

server() ->
  Pid = spawn(fun()-> start() end),
  Pid.

start() ->
  io:format("Started Server:~n"),
  {ok, Socket} = gen_tcp:listen(4001, [binary, {packet, 0},{reuseaddr, true},{active, false}]),
  accept(Socket).

accept(ListenSocket) ->
  io:format("Accept Server:~n"),
  case gen_tcp:accept(ListenSocket) of
    {ok, Socket} ->
      Pid = spawn(fun() ->
        io:format("Connection accepted ~n", []),
        enter_loop(Socket)
                  end),
      io:format("Pid ~p~n",[Pid]),
      gen_tcp:controlling_process(Socket, Pid),
      Pid ! ack,
      accept(ListenSocket);
    Error ->
      exit(Error)
  end.

enter_loop(Socket) ->
  %% make sure to acknowledge owner rights transmission finished
  receive ack -> ok end,
  loop(Socket).

loop(Socket) ->
  io:format("Loop Server:~n"),
  case gen_tcp:recv(Socket, 0) of
    {ok, Data} ->
          io:format("Server got data = ~s~n", [Data]),
          Match = re:run(Data, ".*Emulator-Command:\s(.+)\r", [{capture, [1],list}]),
      case Match of
        {match, ["timeout"]} -> Response = timeout("timeout");
        {match, ["protocolviolation"]} -> Response = protocol_violation("Omitting carriage return, this should cause the protocol violation");
        {match, ["httperror"]} -> Response = http_error("500", "Internal Server Error");
        _ -> Response = response("testing")
      end,

io:format("Match: ~p~n", [Match]),
          gen_tcp:send(Socket, Response);
          %%loop(Socket);
    {error, Reason} ->
      io:format("Error on socket ~p reason: ~p~n", [Socket, Reason]),
      gen_tcp:close(Socket)
  end.

response(Str) ->
  B = iolist_to_binary(Str),
  iolist_to_binary(
    io_lib:fwrite(
      "HTTP/1.0 200 OK\r\nContent-Type: text/html\r\nContent-Length: ~p\r\n\r\n~s",
      [size(B), B])).

timeout(Str) ->
  timer:sleep(1000000000),
  B = iolist_to_binary(Str),
  iolist_to_binary(
    io_lib:fwrite(
      "HTTP/1.0 200 OK\r\nContent-Type: text/html\r\nContent-Length: ~p\r\n\r\n~s",
      [size(B), B])).

protocol_violation(Str) ->
  B = iolist_to_binary(Str),
  iolist_to_binary(
    io_lib:fwrite(
      "HTTP/1.0 200 OK\nContent-Type: text/html\nContent-Length: ~p\n\n~s",
      [size(B), B])).

http_error(Code, Str) ->
  B = iolist_to_binary(Str),
  iolist_to_binary(
    io_lib:fwrite(
      "HTTP/1.0 ~s ~s\r\nContent-Type: text/html\r\nContent-Length: ~p\r\n\r\n~s",
      [Code, B, size(B), B])).

This Erlang module expects you to sent an extra http header Emulator-Command: * where * can be:

httperror -> returns 500 and Internal Server Error
timeout -> runs a long sleep on Erlang side, causes timeout on a proxy side
protocolviolation -> As I built the proxy in ,net WCF it was funny to see that MS takes only \n in the http headers as a protocol violation, so \r\n is required to keep MS happy.

anything else there or omitting this header will just return proper http response with "testing" body.

Disclaimer. I'm only starting to learn Erlang so please take the code with caution of course!

To compile in the erlang shell:

c(httpemulator).

To start emulator server:

httpemulator:server().

To end the emulator process:

exit(pid(0,96,0), old).

Pid is returned at the server start.

Hope it can be useful :).

xcode - Configure for analyzing. The scheme is not configured for analyzing.

2016-04-14T13:59:00.000+02:00

I'm used to run analyzing cycle on my apps when moving closer to their public release. This time xcode (7.3 (7D175)) greeted me with:

"Edit the scheme to enable analyzing, or cancel the action." with that "Edit Scheme ..." was not opening anything and I didn't notice anything extra in the schema editor to enable analyzing.

I even created a clean new project to see if error would be present there as well, and it was.

So after scratching my head a bit I just set a RUN_CLANG_STATIC_ANALYZER as true for the target/build where I needed it:

This way I got the static analysis messages back.

Hope this can help someone!

Update. Or as Jaime Santana commented, just "launch Analysis from the menu option Menu -> Product -> Analyze". Thank you Jamie, that worked!

Note to self - concatenating rows into string with MSSQL

2016-02-18T14:56:00.000+01:00

Here is a sample code:

select count(a.v2_icid), a.v2_icid,
(select distinct(fullname) + ',' AS [text()] from systemuser u
join account acc on acc.ownerid = u.systemuserid
where acc.v2_icid = a.v2_icid For XML PATH ('')) [Users]
from account a
where statecode = 0 and a.v2_icid is not null
group by a.v2_icid
having count(a.v2_icid) > 1
order by a.v2_icid

So, here is my string dump for x64 in windbg

2015-12-07T15:11:00.002+01:00

Based on this and that.

$$>a<"c:\_install\dumpstringtofolder.txt" 000007feed816500 1000 c:\temp\stringtest

and the script:

$$ Dumps the managed strings to a file
$$ Platform x64
$$ First argument is the string method table pointer
$$ Second argument is the Min size of the string that needs to be used filter
$$ the strings
$$ Third is the path of the file
.foreach ($string {!dumpheap -short -mt ${$arg1} -min ${$arg2}})
{
r@$t0= poi(${$string}+8)*2
.writemem ${$arg3}${$string}.txt ${$string}+c L? @$t0
}

The only other thing I needed this time is this on one of the strings:

!gcroot 0000000100e769f0

To see:

-> 0000000180006d08 System.Web.Caching.CacheMultiple
-> 00000001800069c8 System.Web.Caching.CacheCommon
-> 00000001800034a8 System.Web.RequestTimeoutManager
....
-> 00000004021b5740 System.Web.SessionState.InProcSessionState
-> 00000004021b5288 System.Web.SessionState.SessionStateItemCollection
....
-> 0000000289779a58 ASP.ordercomposer_orderedit_aspx
-> 000000018c14fe38 XYZ.Ordering.Modules.OrderingViewStatePresister

So this time the root of the problem was in keeping viewstate in the session and not really purging not needed view state away. This was one of the legacy apps and reminded everyone the old none-tasty view state solution from MS :). Viva MVC and looking forward the day with no legacy ASP.NET apps we need to support :)!

NGINX + PostgreSQL + PostGIS - performance baseline/diff

2014-05-20T23:13:00.001+02:00

Just wanted to share some of the initial benchmark from http_load on the project I'm working right now. Why the excitement? I've been an MS oriented guy for the last 20 years and moving away from IIS and MSSQL (where you can be waiting 20-30 seconds for the initial IIS/.NET/SQL app startup) to NGINX with PostgreSQL stack is life changing - no exaggeration!

So here is what I got on Digital Ocean lowest spec machine (1 core, 20GB SSD, 512 MB RAM, Ubuntu 12) with a bare NGINX:

34178 fetches, 10 max parallel, 683560 bytes, in 5 seconds
20 mean bytes/connection
6835.6 fetches/sec, 136712 bytes/sec
msecs/connect: 0.0445469 mean, 0.937 max, 0.021 min
msecs/first-response: 1.41242 mean, 101.975 max, 0.257 min

And with the full nginx->postgresql with postgis searching for speed limits around submitted location (but only a few limits present in db for a moment):

5574 fetches, 10 max parallel, 6.22058e+06 bytes, in 5 seconds
1116 mean bytes/connection
1114.8 fetches/sec, 1.24412e+06 bytes/sec
msecs/connect: 0.0561055 mean, 0.37 max, 0.023 min
msecs/first-response: 8.68285 mean, 187.733 max, 0.812 min

So it got from almost 7k to 1k per second, still that's from the different universe compared to what IIS/.NET would do for me, and look at the machine spec! Plus, $0.0 for nginx + $0.0 for posgre/postgis - pure win!

-- Stan.