Rack my brain!

How to integrate AVM smart devices into HomeKit by using openHAB

2019-05-08T22:13:00.000+02:00

After some evaluating openHAB (Vers. 2.2.0) on my NAS I wanted to do it right and started with this (German) article to use openHABian together with a Raspberry Pi.My primary goal was to integrate my AVM smart home devices to control them with HomeKit.

To get started I used the following items:

Raspberry PI
SD Card 16 GB

And the following software (I used a Mac, so you might need to replace some tools if you use another OS):

Download the latest openHABian image here. This is a dedicated linux distribution to provide an openHAB server.
Download SD Formatter here. Needed to format your SD card.
Download Etcher here. Needed to install the openHABian image on the SD card.

Installation openHABian

Insert the SD card
Connect the Raspberry Pi via LAN with your network
Plugin the Raspberry Pi
Wait about 45 minutes until all updates were installed
When everything went well you should be able to connect to http://openhabianpi:8080
Connect via ssh: ssh openhabian@openhabianpi (standard password: openhabian)
Open the config: sudo openhabian-config
Run 01 "Update"
Run 02 "Upgrade System"

Configuration openHAB

Open http://openhabianpi:8080/start/index
Install AVM FRITZ! Binding
Install HomeKit Add-on

Configure AVM Things

First of all you need to connect the Fritz!Box, therefore you should get a notification in the inbox in PAPER UI:

https://openhabianpi:8443/paperui/index.html#/inbox/search

You just need a Fritz!Box user. I created a dedicated one which only has the permission "Smart Home". HTTPS is only available when you MyFRITZ! Service for remote access. When the Fritz!Box is connected all other connected smart devices become visible:

Configure HomeKit Add-on

You can keep all the defaults, but you need to add the IP for the interface. In my case this is just the same IP as the Raspberry Pi has, my PI just has one network interface.

If you do not add the IP here your device will be listed in the Home app but paring will always fail, as described here.

Create HomeKit Items

To use your items with HomeKit you need to create so called "accessories" as described here. As follows a little sample:

Switch HK_Livingroom_Light "Light Livingroom" <light> (gLightLivingroom) [ "Lighting" ] { channel ="avmfritz:FRITZ_DECT_200:10_51_2_165:667777888999:outlet" }

Number HK_Livingroom_Temperatur "Temperature Livingroom" (gTemperatureLivingroom) [ "CurrentTemperature" ] { channel ="avmfritz:FRITZ_DECT_200:10_51_2_165:667777888999:temperature" }

To copy the items I use the following command:

scp /Users/philipp/Desktop/HK.items openhabian@openhabianpi:/etc/openhab2/items/HK.items

Start openHAB service manually

Type in the following command to start the service manually:

sudo systemctl start openhab2

Start openHAB within terminal session

1. openhab-cli stop

2. openhab-cli clean-cache

3. openhab-cli start

This will start the service and also the Karaf console. When you close your SSH session the service will also go down.

To just open the Karaf console for your existing instance use:
openhab-cli console

Start openHAB service automatically after reboot

sudo systemctl enable openhab2

sudo shutdown -r now
sudo systemctl status openhab2

Use the script above to enable automatic start after reboot and to test it.

Permission Fix

For some reason the permission was messed up and I had to fix it with sudo openhabian-config.

Trace & Debug issues

To trace issues with the HomeKit integration I enabled trace for it by submitting this command within the Karaf console:

log:set TRACE com.beowulfe.hap

To change it back I used

log:set DEFAULT com.beowulfe.hap

HomeKit Pairing

I paired my openHAB with the iOS Home app.

Open the app
Add new device
Choose "Code is missing or cannot scan" below
Here the openHAB instance should be listed
Select it and type in the code you defined in openHAB's HomeKit add-on configuration.
Pairing should be successful now.

Reset HomeKit Pairing

ssh -p 8101 openhab@localhost (not needed with openHABian anymore)
smarthome:homekit allowUnauthenticated true
smarthome:homekit clearPairings

Add SSH Key

To create and upload my SSH key I used a Mac. So you might need to adapt your script to do it with another operation system.

Create SSH key: ssh-keygen -t rsa
Copy public key to the Raspberry Pi: ssh-copy-id openhabian@openhabianpi
Type in the passphrase so you do not need to type it in again: ssh-add

On a Mac you can create a bookmark. Therefore you need the following commands:

1. To create the needed config file: nano ~/.ssh/config

2. To create the bookmark you need the following lines:

Host openhab

User openhabian

Port 22

HostName openhabianpi

IdentityFile ~/.ssh/id_rsa

3. with ctrl + x you leave nano and you will be asked to save the file

4. Then you can open the terminal with ssh openhab without typing in a password

5. With passwd I changed my password for the user openhabian to get rid of the default password

Add Fritz!Box certificate (Update 2020-04-25)

scp /Users/myuser/Downloads/boxcert.cer openhabian@openhab:./fritz.crt
sudo find / -name jre
cd /opt/jdk/zulu8.40.0.178-ca-jdk1.8.0_222-linux_aarch32hf/jre/bin
sudo ./keytool -importcert -alias fritzbox -keystore /usr/lib/jvm/zulu-8-amd64/jre/lib/security/cacerts -file ./fritz.crt

Then it is possible to configure your Fritz!Box with https:

How to move iTunes library to NAS folder

2019-03-03T15:14:00.001+01:00

This is nothing new, but to free up some space on my MacBook I decided to move my iTunes library to a network drive on my NAS. I got some inspiration from this old German video. I tried to collect all relevant information for doing so.

I did it by following this steps

I closed iTunes
I mounted a network drive with AFP
I copied my iTunes library folder to a folder on the network drive (it is important to keep the folder name "iTunes"!)
I opened iTunes while pressing the "alt" key. Now you can create a library or choose one from another location
I set the new iTunes folder
I deleted the local iTunes library

My iTunes settings look like this now, iTunes took over the selected folder:

Keep in mind the following

You cannot open the iTunes library with two accounts at the same time.
When you choose to set "Keep iTunes Media folder organized" iTunes will set your folder structure for artists an albums and also the file pattern for your songs. You have to think about that, because iTunes touches and changes the files again and again (for example to adjust the metadata).
iTunes library folder and music folder can be separated when "Keep iTunes Media folder organized" is disabled. Then you can just add files from another location and it will be kept there without making a copy to the iTunes library folder.
One disadvantage for keeping the folders separated can occur with albums which you bought from iTunes you get song duplicates. This problem is dragged along when syncing music to iPhone.

How to delete Azure Active-Directories

2016-02-21T22:49:00.000+01:00

When I was trying to delete an Azure Active-irectory by using the management portal I have received the error message “Directory contains one or more applications that were added by a user or administrator” and I was really confused.

Then I found out this is a common issue. The solution was even more confusing, some PowerShell cmdlets are needed. Unfortunately I was not able to proceed, it was not possible to login with my Microsoft-ID to execute the needed commands.

Here I found out I have to install the correct version of the tools. These are located here.

Install and verify IIS HttpModules with custom config sections

2015-02-01T19:13:00.000+01:00

A week ago I started to learn about HttpModules and HttpHandlers watching a pluralsight course by Robert Boedigheimer. First of all I want to really recommend this course, I learned a lot of stuff which I could not find in any other course, book or blog post.

As I already did in the past with another course, I want to add some comments and extend the HttpModule part. I refer to the course example with the serverMaskModuleGAC.

I walked into some trap doors and thereby I found another approach to simplify the installation process of the module to GAC a little bit. With this approach I am able to verify my custom config sections and module registration is correct.

Here is my approach:

1. When you have finished your implementation for serverMaskModule, install it in the GAC as usual by using the gacutil tool. By the way, there are some traps when you need to install it on Windows Server 2012. Here I am using Windows 8.1 64 Bit. I created my HttpModule with .NET 2.0 because it seems to be more compatible with different IIS and Windows versions. On Windows Server 2012 I was not able to install it with .NET version 4.5.

2. Copy your custom config section schema into the folder C:\Windows\System32\inetsrv\config\schema

3. Add the custom config section in applicationHost.config

4. Register your module by using IIS Manager. Click on the server node, and choose Modules:

5. Click "Add Managed Module"

6. Click again on the server node and choose Configuration Editor

7. Choose the serverMaskedModule section

8. For testing purposes choose another value for header filter (In my implementation I have changed the parameter name from "name="serverMaskModuleGACHeaders" to "headerFilter"):

As long as the value is shown in normal print, it is the default value coming from the custom config section. When you change the value it will be bold printed, IIS Manager will add the config section in the applicationHost.config file.

9. If you did not get any error messages so far you can start sending your first test requests to verify the module is working and filtering the headers you have chosen in the in the serverMaskModule section.

The advantage of this approach is you can test and verify, that module registration and your custom config is validated. E.g. when you have a spelling mistake in your config section you will get an error messages like this:

How to install DLLs to the GAC on Windows Server 2012

2015-02-01T15:06:00.000+01:00

Currently I am collecting interesting links about it. Is there a better solution?

How does AppFabric cache calculates the values for RequestCount, ReadRequestCount, WriteRequestCount and MissCount?

2014-06-29T11:43:00.000+02:00

When I started to use the Get-CacheStatistics cmdlet to monitor caching activities more intensivley I was often confused. Here an example:

Should not be ReadCount+WriteCount = RequestCount? Why is WriteRequestCount increasing +2 although I just send one put request?

By talking to Microsoft support I figured out how AppFabric cache calculates its numbers (Thanks to Gayathri from Distributed Service Team). Number calculation behavior depends on your configuration, high availability and local cache have a major impact on it.

Activity	RequestCount	ReadRequestCount	WriteRequestCount	MissCount
DataCacheFactory is initialized and Named Cache is up	+1	-	-	-
Put item, not yet in local and server cache	+1	-	+1 (+2 with HA)	+1
Get item from local cache	-	-	-	-
Get item from server cache, local cache has expired	+1	+1	-	-
Get item, local and server cache has expired	+1	+1	-	+1

I hope this is helpful for you when you are analyzing the caching behavior for your application.

Clean up AppFabric Dashboard Part 2: Duplicate collection entry in Web.config

2014-03-20T20:18:00.000+01:00

In Part one I have already described how to remove phantom apps here on more supplement. These phantom apps could also remain in the AppFabric Monitoring db:

Just delete no longer needed entries in this table.

Duplicate collection entry

When you receive this kind of issue, like "[...] Cannot add duplicate collection entry [...]" you have defined the a configuration section twice in two different Web.config files in different places in your file system.

Clear

One easy solution is to use a clear statement.

Searching for the inherited element

There is a post in Microsoft's documentation which is explaining how to do it.

Visual Studio in debug mode

When you are working with Visual Studio in debug mode, which means you are just pressing F5 to run it on your local IIS instead of publishing it to a web server, you should keep in mind that e.g. your obj folder might contain more copies of your Web.config file. This could lead into this duplicate collection issue as well.

Cleanup AppFabric Dashboard Part 1: Old virtual paths and phantom applications on IIS

2014-03-20T08:05:00.000+01:00

While you are working with IIS installing and uninstalling web applications you might run into this issue on AppFabric Dashboard:

Another example is this error message:

There are two steps necessary to get a cleaned AppFabric Dashboard.

Delete old virtual paths

Old virtual paths might remain in %windir%\system32\inetsrv\config\ApplicationHost.config. To get rid of these crappy virtual paths switch to command line tool and use the following script:

C:\Windows\System32\inetsrv>appcmd.exe list vdir

VDIR "Default Web Site/" (physicalPath:%SystemDrive%\inetpub\wwwroot)
VDIR "Default Web Site/Service/App1/" (physicalPath:C:\Service\App1)
VDIR "Default Web Site/Service/App2" (physicalPath:C:\Service\App2)
VDIR "Team Foundation Server/" (physicalPath:C:\Program Files\Microsoft Team Foundation Server 12.0\Tools\Templates)
VDIR "Team Foundation Server/tfs/" (physicalPath:C:\Program Files\Microsoft Team Foundation Server 12.0\Application Tier\Web Services)
VDIR "Team Foundation Server/tfs/queue/" (physicalPath:C:\Program Files\Microsoft Team Foundation Server 12.0\Application Tier\Message Queue)

C:\Windows\System32\inetsrv>appcmd.exe delete vdir "Default Web Site/Service/App1/"
VDIR object "Default Web Site/Service/App1/" deleted

Remove phantom applications

The file might also contain some obsolete application entries. I found the solution here.
Search for application XML nodes in your applicationHost.config file and delete them if the the application does no longer exist. Of course, do not forget to create a backup of the file before you start changing it.

To prevent this issue

Always remove deployments like this to avoid, or at least reduce the risk to run into this issue.

Two annoying issues in Visual Studio regarding XSDs and WCF should be fixed immediately

2014-03-01T22:24:00.000+01:00

In my daily work I am always stumbling on two annoying features in Visual Studio, and I think it could be improved easily. Here is what I am complaining about.

WCF Test Client

WCF Test Client might be a nice tiny tool for small WCF tests, especially when you are testing smaller scenarios or when you are starting to learn how to implement web services with .NET. But it can be very annoying when you are working on more advanced stuff and your are testing with your local IIS. WCF Test Client always(!) starts automatically when you press F5 for debugging and your opened .svc.cs file has the focus:

When I have configured local IIS is the target for debugging, then I expect my application will start with local IIS and not with WCF Test Client. Always.

Here is my first urgent request: Please give me an option to disable WCF Test Client completely!

Update 03.03.2014: Workaround for WCF Test Client Issue

Select project file -> Right click -> Properties -> Web -> Select Don’t open a page.

XSD Design Start View

When you open XSD files, which are part of your solution file it will always be opened with XSD Design Start View:

And here is my second urgent request: Please give me the option in the settings to always jump into the plain text file without jumping into this XSD Design View first!

Update 03.03.2014: Solution for XSD problem

The solution can be found here:

XSD: Right-click -> Open With.... Then select the XML Editor and hit Set As Default

.NET Programming: Jumpstart ETW (Event Tracking for Windows)

2014-03-01T18:47:00.000+01:00

To get started with ETW I highly recommend this pluralsight course by Kathleen Dollard. It gives you a really good introduction, including background and some examples. It will be much harder to succeed just by googling or reading books.

I will not spoil anything from this course. Here I will just give a few more hints and annotations to show where I struggled with ETW, to help you not doing the same mistakes. After finishing the pluralsight course I was really eager to get going with my own implementation but I ran into some annoying trapped doors.

Nuget package for EventSource

The current nuget package for EventSource (Vers. 1.0.16) will create the manifest file automatically (it validates your implementation upfront, only when validation succeeds manifest file will be created. Otherwise there will be no updated or no new manifest file in your bin folder!). Therefore there is no need to create it by hand.

wevtutil.exe

Running wevtutil.exe without parameters to see its help, produces this output on my console:

But all necessary commands are working. I am still investigating into this issue. But it will not effect the needed functionality, all operations worked fine for me.

In the beginning I had some trouble with wevtutil.exe

Making spelling mistakes
Specifying dll file instead of man file
And wrong paths (copy paste issues)

Doing these mistakes will lead into error messages like this:

Seeing these error messages for the first time confused me:

`At column=0, The system cannot locate the resource specified. Failed to load xml document`

The message made me thinking about something was wrong in my implementation in my manifest file. Maybe wrong or missing resources for translation, something in this direction. But its just talking about the parameters for wevtutil and indicates you are specifing a file that is not existing (as already described above spelling mistake in one of the paths or specified dll instead of man file and so on).

EventSource names

I had some issues specifing a “valid” EventSource name. I specified a name like this “MyCompany-MyApplication-MyEvents”. I was wondering why the regarding nested folder was not created. I could just see this entry far in the bottom in the Event Viewer:

But when I tried to open the regarding log I received this message:

The solution was really simple. There was already an existing Eventlog with the name “MyCompany” created by another application:

In this case you cannot create a nested folder with the same name. Makes sense, but a better error message would be helpful.

Maintenance for builds and installed manifests

Renaming and building EventSources will create new dll and man files in your build folder but it will not remove the old files. I always delete the content of my build folder when I do changes to keep the overview.
You need to keep your custom event sources (dll files) in installation folder (when you remove it, your views in Event Viewer will look strange), you better create a suitable folder for it. Keep man files there too, to be able to uninstall the events (I could not find a way to remove my custom events without man files).

Last but not least

Finally you should definitely checkout this nuget package with more EventSource examples.

How to get started with Windows Server AppFabric easily

2014-01-29T19:25:00.000+01:00

In my opinion it is really hard to get started with Windows Server AppFabric. There is good material, but it is shuffled around in different sources. A good intro can you find here:

this all-in-one intro by Scott Hanslmann
this caching intro by Wade Wegener

Additional helpful resources are

Good books are

Please also watch out for software updates, currently there is CU 4.
To keep yourself updated subscribe to Distributed Services News Feed.

Updated my list on 11.03.2014 with one more

Download the Windows Server AppFabric Samples

How to fix Windows Server AppFabric Dashboard on Windows Server 2012 / Windows 8

2014-01-28T19:42:00.000+01:00

Windows Server AppFabric is a great enhancement for IIS and its caching capabilities are great for .NET applications. But currently there is a very annoying bug regarding its monitoring UI, so called “Dashboard”. Especially for me as a non WF guy (I am actually just working with WCF) it was really hard to figure it out. Hence, many thanks to Gianluca from Distributed Services at Microsoft!

Windows Server AppFabric Dashboard does not work on Windows Server 2012 / Windows 8. You will receive this error message "Unable to cast object of type 'System.Management.Automation.PSCustomObject' to type 'Microsoft.ApplicationServer.Management.Data.GroupByInfo'." This error will cause that you cannot see any request statistics neither WF nor for WCF:

The reason is, I quote Microsoft here:

"[...] the root cause is a powershell script embedded in AF code that returns different results when ran on Windows 2012. This script is about statistics for Idle & persisted WF instances, if the related table in the SQL DB is empty then the script throws the exception that you see in the error message. With just 1 idle or persisted WF the script runs to completion without any errors [...]"

The table is called "InstancesTable" and it is placed in the AppFabric Persistence database.

Actually there is no hotfix for this issue (Microsoft says we can expect it with CU5). To fix it follow the following steps:

1. Download this folder (source code can be found in the sub folder, hosted on OneDrive)

2. Deploy\Import the WF application "WorkflowService1.zip" on your IIS.

3. Execute "TestIdleWF.exe". You screen should look like this:

4. Check the event viewer in the section "Applications and Services Logs" => Microsoft => Windows => "Application Server-System Services". Here you should not find any issues regarding connection to your AppFabric Monitoring and Persistence databases. Otherwise you have to fix your SQL Server Connection Issues first and start again with step 3.

5. Check the table "InstancesTable" in your AppFabric Persistence database, it should contain at least one line now:

If the table is still empty you need to configure the persistence store for your application. Right click on your application -> Manage WCF and WF Services -> Configure -> Workflow Persistence:

6. Now you should be able to look into AppFabric Dashboard and see your requests now. For views where only WCF service history is displayed you will still find the known error message, but it can be ignored, Dashboard will work as expected. For views with WF or WF and WCF the message will disappear and reporting will be shown:

Update: Cumulative Update 5 will fix this issue!

Did you ever think about two-step verification to increase your password security?

2014-01-28T19:21:00.000+01:00

My feeling is that two-step verification is still not very common, although a lot of services are listing this feature. Here you will find them:

Facebook
Google
Microsoft
Twitter
Evernote
Dropbox
Apple (currently just available in U.S., UK, Australia, Ireland, and New Zealand)

Helpful apps are e.g. Google Authenticator or Authy.

Updated my list on 27.02.2014 with some more services

Buffer
Yahoo!
Apple added, Canada, France, Germany, Italy, Japan, Spain

Updated my list on 10.03.2014 with one more

Hootsuite

Updated my list on 26.03.2014

tumblr

Updated my list on 02.04.2014

Why I do not want to use another iOS podcast client than Castro

2014-01-28T05:35:00.000+01:00

Castro is still pretty fresh in the iOS App Store (currently version 1.0.6, first release came in late 2013) but has already beaten the competitors in my opinion. Here is why:

Castro is winning by simplicity. Focusing on features and using using the fresh iOS 7 design.
Great simple feed search engine (worked perfect for my favorite German podcasts).
Push notification works.
It is stable.
Once again, I like the simple intuitive design.

Other settled podcast player in iOS market help Castro to succeed by not fixing their issues. I am coming from Instacast and this is why I think it does not fit anymore:

When I open Instacast by sliding push messages on my lock screen it crashes nearly up to 50%.
Up Next Feature does not work for me. Lists are useless, everything works just through Up Next.
Sync feature with its OS X app is a mess, similar to Apple’s Podcast app.

This is why Apple’s Podcast app does not work either:

Sync with other iOS and OS X devices is a mess.
Push notification is not existing (Is it? I have never seen any).
Navigation is too complicated.

By using Castro I completely discarded thinking about syncing podcasts on several devices, I am using Airplay devices like AppleTV or AirServer.

My ideas to improve Castro:

Add an option for continuously playing all my podcast episodes, instead of stopping after each one.
Add support for password protected feeds.
Start playing an episode immediately by opening the episode. I do not want to tap play after opening it.
Integrate Airplay button in the lower control bar.

Update:

Add missing sharing options to share your current podcast episode on Twitter.
Double Tap to play episodes immediately instead of opening it.

Have you ever reconsidered your personal iPhone security policy?

2014-01-19T17:57:00.000+01:00

Why should I reconsider my iPhone security policy? I have my iPhone four digit pass code and Find my iPhone feature feature is activated. So how should abuse be possible? What could a thief be doing with my stolen iPhone? Well, possibly quite a lot. At least the bold print bullets in this article should be mandatory for everyone. Of course this is no guarantee for invulnerability but it will improve your security. After watching this video you should consider the following actions:

Change your Apple ID rescue mail address and do not add this mail account to your iPhone.
Consider to use a proper and usable password policy for your Apple ID like this or this (German article).
Consider a complex pass code for your iPhone. As you can see in the video four digit pass codes can be hacked in no time on iPhone 4. For newer iPhone a leak is not yet known but it is properly there. Definitely do not use these pass codes.
Alternative to point three: activate delete iPhone after 10 wrong attempts (Go to Settings > Passcode Lock > Enter Your Passcode > Click on Done > Erase Data = ON > Enable). This feature can be very annoying when friends playing with your phone at a party ("Haha, I erased your iPhone!").
Deactivate control center when iPhone is locked. I agree that it's a little uncomfortable.
There is a long still ongoing discussion about the pros and cons of Apple's Touch ID which is build in the iPhone 5S. No matter what it is definitely more secure than any 4 digit pass code.

After reading this (German article) you should consider the following:

Upgrade to iOS 7 if not already done.
Reinstall your apps after upgrading to enable data protection, at least apps with sensitive data. Otherwise third party app data might be there in plain text. (Just updating the app under iOS 7 will not activate data protection, you need to reinstall the apps).

After seeing this video (iPhone part starts at 44:30) you might consider to

Do not to use mobile phones at all. Maybe not an option for most of us.
Block your phone camera (at least your front cam) with a sticker. There is no evidence yet, that somebody can access it but it does not hurt to cover it. And we already saw this issues on PC's and notebooks here and here.

After reading this (German article) you might consider to

Do not use Key Chain Sync.
Do not use iCloud Backup, use local backup instead.
Check your privacy setting and location services.