Amro’s Blog

Facilitating Instead of Dictating

2021-09-22T07:27:09-07:00

Early in my career I thought showing my worth on a team meant having my ideas win. I was far too attached to those ideas, which left me frustrated when they weren’t chosen and worst of all made working with others more difficult. Over time I learned the right approach wasn’t to dictate solutions, but to facilitate problem solving. This approach has benefited everyone involved, led to better solutions, and to better relationships.

Being a Dictator #

Dictating solutions, be it as an engineering lead or a product manager or from entirely outside a team, creates a single point of failure and stifles the creativity of one’s team. The dictator becomes a bottleneck for the team, which slows down decision making, robs others of a chance to grow and contribute, and ironically stunts the dictator’s own growth as they spend time answering questions for others rather than looking ahead. Dictating solutions also alienates others and harms working relationships when mixed with poor communication or aggression.

Often this sort of behavior isn’t malicious, the dictator may even believe they’re helping. People inadvertently cause harm in an attempt to be the go-to person, doing their best to remove a bit of the burden from others. That’s what I thought I was doing many years ago, and it wasn’t until I saw others facilitate that I realized the benefits of intentionally making space for others to contribute.

Being a Facilitator #

Leading as a facilitator means coming to one’s team with a problem instead of dictating a solution, then working with them to find the right approach regardless of whose idea it is. This shows the team their input is valued, which helps them become invested in the work and it makes space for others to learn, teach, and show their creativity.

Sometimes there’s a lack of ideas because the problem space is poorly defined or too big. In the former case I like to ask questions to help better frame the problem, and in the latter I like to break the problem down into smaller problems to be solved. If there are many problems I like to explicitly delegate finding a solution to each problem to a single person. They don’t have to figure it out on their own, but they’re responsible for coming back to the group with a solution.

Being a facilitator also means admitting one’s mistakes and making peace with letting go of good ideas in the interest of moving the team forward. In the past I’ve made the mistake of coming to my team with a solution, and even when it was a good solution that action left them wondering what their role was aside from execution. This behavior is sometimes displayed by engineers who transition into management roles—that was certainly the case for me. Solving engineering problems was something I was comfortable doing and it was (and sometimes still is) easy to fall into wanting to overstep in the interest of being helpful. That’s a great way to waste talented minds and prevent people from learning. This isn’t limited to new engineering managers though—I’ve observed this behavior at various levels of leadership.

These days I’m better at making space for others to thrive, and it comes from one simple question: am I coming to my team with a solution or a problem? I’m still a problem-solver at heart, so when I do have a suggestion I make sure to frame it as an option, one option, something to consider and not a prescription. Of course, sometimes I do need to step in, and it’s a fine line, but I try to do so by asking questions to ensure we consider key problems we need to solve. That’s because, as always, it’s about the problems, not the solutions.

Delivering Difficult News

2021-08-23T03:43:05-07:00

Earlier this year I was asked if some engineers from our team could help build a newly prioritized project. Doing so meant putting our team’s current work at substantial risk of failure. I knew canceling our project was the right thing to do because the incoming request was more important and because continuing our current work meant spreading our team too thin. Our team had put in a lot of time so I wanted to share the news empathetically, but delivering and receiving difficult news is never easy.

I started by talking to my product manager. They quickly understood and helped me plan how we would deliver the news to our team. We drafted a list of points to communicate. Having a list was important because we didn’t want to leave out part of the message when speaking in different places or with different people. We detailed what was changing, why, what would happen next, and honestly that we made the decision. It was important to include we made the decision to avoid an “us vs. them” situation with the people who asked for our help. We shared the plan with our managers, then moved on to telling the team. We wanted to tell them as soon as possible out of respect and so they wouldn’t hear an incomplete message through some other channel.

Having drafted the message we scheduled a 15 minute chat with each person on the team. These meetings took a full day and this process was slower than telling the team all at once, but it gave them a place to ask questions openly without the pressure that might come with speaking up in a bigger meeting. We took two or three minutes to share the message, and spent the rest of the time answering their questions earnestly. We gladly spent more time with people who had additional concerns as we wanted them to know we cared about them and their work. One person requested a retrospective, which we committed to having later in the week.

Later that day we had a broader team meeting where we repeated the news the same way and answered questions from the group. This let the team settle into the decision together. By now the news wasn’t new to anyone, which meant the collective response was not one of shock. The people that initially requested our help attended the meeting so they could answer questions and explain why the new project was important for our users and product. And of course we held a retrospective later in the week, as promised.

After the team meeting we sent a brief email with the same message to our team and adjacent people working with us. The idea was to ensure people who couldn’t attend earlier meetings received the message.

While our team was disappointed that we cancelled the project, they understood the circumstances and took the news well. More than one person told us they appreciated how we communicated the cancellation. We helped with the other project, and worked to deliver a new, well received, project from our team some time later.

It’s worth planning how you communicate difficult news. Our approach was to be honest, clear, and consistent. We were quick to communicate, accountable for the decision, gave everyone a chance to ask questions in a low pressure environment, repeated the message consistently in more than one place, and gave people multiple chances to share their concerns. We then worked to address them where we were able to do so.

Follow @amdev on Twitter for occasional management and engineering thoughts, but mostly puns and pizza.

It's About Managing Expectations

2021-08-09T07:41:47-07:00

Some time back my team was asked to take up a critical project on short notice. My product manager and I found out about the project on Monday evening and we were told it needed to ship on Friday the same week to 3 client platforms with a small team. We had just four days to make this happen and the date couldn’t move so we had to work with the other two project levers: changing the amount of work we would deliver, and putting more people on the project. The latter famously only helps so much. I want to share how we got many people on the same page and kept them there by managing their expectations.

What are expectations?
Expectations are things people believe will happen in the future. Unmet expectations can lead to unfortunate outcomes like duplicate effort, missed opportunities, or worse. Managing expectations is the act of making promises you intend to keep, making sure other people are on board with the plan throughout the process, and bringing them along when things change. Managing expectations helps us avoid unfortunate outcomes, sets a baseline for accountability, and builds trust that can lead to new opportunities.

Planning
The first step my PM and I took was to get each other on the same page. That night we discussed our approach and how we’d divide the work. We also decided we would not block on each other when making decisions. This helped us move more quickly and cover for each other when issues came up. We wrote a plan early Tuesday morning detailing our recommended approach and the timeline on which we would deliver the project. We planned just one and a half days for engineering and half a day for QA. That meant we’d ship Friday on one platform and branch for release on the two others that required a longer release process. We chose this approach so we could understand the effect of our change in one market before going global with it. This gave us confidence and helped us get buy-in from a bunch of folks. We deliberately chose the smallest amount of work we could do that would meet our goals. We also let our team know something disruptive was coming and that we’d share more as soon as we could.

Our plan included other solutions we considered and why we had ruled them out. We did this to avoid spending time going back and forth negotiating what we’d build. In this case our primary stakeholders were a few decision makers farther up and peers from other teams, some of which our change might affect. We made sure we had the go-ahead from the former, but later found out two peers had concerns about the plan. Given the tight timeline we’d gone for the simplest option, which was to turn off their unshipped features temporarily when our temporary change was enabled. We talked with our peers and shared our constraints with them, then agreed to keep their features enabled on a best-effort basis. Their features were complimentary so this was the right thing to do in the end.

My take-away from this was it’s important to check my assumptions about the expectations others have of me and my team because approval from decision makers isn’t the same as a mandate. I now ask myself what expectations I have of others, and what expectations they have of me and I try to answer those questions before moving forward.

Sometimes these expectations are clear, but what if they haven’t been said aloud and agreed upon? In my experience this ambiguity leads to misaligned expectations and in turn the sort of unfortunate outcomes mentioned above. I address this by asking questions that might seem silly in my mind and by stating my understanding of our shared expectations aloud. I’ve found others often have the same questions so they end up being not so silly after all. When stating shared expectations I talk about next steps, who is accountable for them, timelines, and expected communication cadence. This gives others a chance to correct our shared understanding. I also ensure decision makers are explicitly onboard with the plan and timeline, not just initially but also throughout the product process, even if the plan does not change. This is important because even if my team’s plan doesn’t change, the plans of adjacent teams or the company might change, and that could affect the relative priority of my work. Another mistake I’ve made in the past is assuming a lack of response to an update was the same as continued approval. I now like to ask for explicit confirmation or feedback on updates so I can work to address concerns if needed.

With everyone on the same page, we shared the plan with our engineering team and asked if they could meet the timeline. This was mid-day on Tuesday. It was critical to have their okay on the plan because they were the ones that would make the changes we had in mind. They said yes, and we told our stakeholders we were good to go. We were now accountable for the work and the timeline. This wasn’t about taking blame if we failed so much as having an understanding of our goals, and knowing we needed to share progress and roadblocks along the way. We gave regular updates on our progress, which in this case was at least twice a day given the nature of the work. We also shared new decisions we made knowing we could adjust later if needed.

Managing Expectations
We’ve talked about how to get everyone on the same page, but we haven’t talked about how to keep them there. That entails communicating with people based on their role with some regularity. Here I ask myself what information the people I’m communicating with need to do their job and how often they need it. In other words, it’s not enough to “manage upward” — you need to manage communication in every direction.

For instance, a high level decision maker likely doesn’t need to know every detail of my team’s project, but they probably want to understand whether progress is being made, whether it will ship when they expect, and if we need help. Our updates to decision makers on this project were on the order of a few bullet points, which we followed by answering any questions they had. Peers might expect more detailed updates relevant to their role.

I mentioned we shared updates at least twice a day on this project, but communication cadence depends on the circumstances. I think about sharing updates at a cadence that keeps my work top of mind, but not so often that I’m not conveying meaningful progress. The idea is to make sure stakeholders don’t feel out of touch with your work. Knowing you’re making progress also helps when it comes time for them to shelve projects that aren’t working out.

You made it all the way to the end of this post so I suppose you want to know how the story ended: we shipped on time with happy stakeholders and our change met the goals we set in our plan.

Follow me on Twitter if you like puns, pizza, and performance reviews.

Software Engineering Isn't Magic

2021-03-13T09:23:41-08:00

The other day I was chatting with a friend who is both very competent and early in their software engineering career. They had noticed it sometimes took others less time to solve problems than it took them, and when they’d ask for help, these folks knew the answer or could often point them in the right direction right away. It felt like magic.

Thinking this is normal, but it’s not magic. When people come back with an answer quickly it’s because they’re aware of the problem, already understand how a system works, or because they’re pattern matching against previous experiences and happened to guess the issue. But if you’ve ever handed off a difficult bug only to have a peer come back the next day with what seems like a simple fix, it can trigger negative feelings.

A couple of years ago I was trying to understand a strange bug in my company’s app that broke the theming system with a new beta release of iOS. After several hours I was able to figure out where our theming code now failed, but I was stuck trying to find the root cause of the bug. I asked the person who wrote the theming system for help and they came back the next day with a single, subtle observation about a behavior change in the underlying system libraries that allowed them to fix the problem. In this moment software felt like magic, but in reality this person had invested a tremendous amount of time learning the underlying system libraries. They’d spent hours debugging the problem to understand it. That isn’t magic — that is expertise and putting in the time.

A year or two before that I was helping improve build times for our team when I hit a segmentation fault in a popular compiler. I remember feeling helpless until a more experienced engineer said something like “it’s just software and we can debug it like anything else¹.” Ah, just because it’s a black box doesn’t mean it’s magic. It’s a system we can debug because we can observe its inputs and outputs. That gave me the boost I needed to go back to basics: I started commenting out code to see where the compiler failed. After a while I was able to pinpoint the problem (two particular operator overloads) and found a solution that let us roll out a major reduction in full and incremental build times.

It’s okay for you to take longer than others to find problems or fix things, and it’s okay to need help from others. You’ll get better at what you do over time too. They’re not faster because they work magic — they’ve just seen this before. Sometimes you’ll see something that others haven’t seen, and suddenly you’ll be the magician.

Thanks to my friend Joe (@mergesort) for helping me edit this post. You should follow him on Twitter. Oh, and I’m on Twitter as @amdev as well.

They meant literally build and debug the compiler ↩

Thoughts on Teams

2021-01-25T20:05:23-08:00

People have different value systems. Someone may value very expensive headphones, or cameras and you may value sneakers. Someone may value working really hard, and you may value having time to live and travel. It’s best to not push your value system onto others, because it can harm relationships. It’s okay and good that people value different things.

Related to value systems — how does one’s value system affect how they weigh tradeoffs? Is this refactor more or less important than shipping? Internationalization? Accessibility? Polish? What stage is your company at? What’s the right thing to do? How does one’s value system help define “right”? Can you think of a way to acknowledge a peer’s values and leverage strengths in a way that helps your team succeed and makes them happy?

Value systems also affect motivations, which vary from person to person. Some people are motivated by public praise, or maybe the rush of shipping something new and exciting, knowing they built something really well, realizing a lot of people will use their work, just helping others, money, or most likely a combination of these things and others. It’s hard to change people’s motivations, but understanding them can help you improve your team’s performance as you work to fulfill them on an individual level. (The team’s needs should come first most of the time, of course.)

Working hard at your job will sometimes net you rewards, but working hard alone isn’t enough. You have to work on the right things — generally the right things end up being whatever is important to the people making decisions. In my experience it’s easier to burn out working hard on the wrong things, but it can also be really gratifying serving unfulfilled needs. The lesson here is to think about your goals before working hard and choose the right things to work on based on those goals and what you value.

Be kind, but also direct with others in business to avoid misunderstandings. Sort of related — don’t be a jerk or you’ll harm relationships and people won’t want to work with you.

Try to be “on” at work. Make a todo list (however you like), and do either the most important thing or the thing you have time to do right now. Mark it off. Remove things that stay on the list for too long without progress and aren’t actually critical.

Let people know where you’re at regularly and set expectations on when you can get back to them with what they need. This is a lot of what it takes to succeed. Once you’re good at your core responsibilities, look for problems that get in the way of your team’s success. Set out to fix those issues, and get help from your peers. Share credit with and publicly thank the people who help. You’ll build trust and get recognition. Take a broader view of “team” over time and repeat this process to drive bigger change.

Is this similar to your experience? Lemme know on Twitter.

Resolving Conflict with Empathy

2021-01-25T20:03:13-08:00

Note: I wrote the following note over a year ago, but only recently decided to send it to my newsletter. I’m not sure why I sat on it so long. Your [respectful] thoughts are welcome on Twitter.

Empathy doesn’t always come easy, but it goes a very long way toward building trust with those around you. I’ve seen many well-intentioned people lose the trust of their peers because they fail to communicate with empathy. Perhaps they rush when replying to a message, or they fail to set aside frustration with a decision they believe should have gone a different way. In many cases this is a matter of differently weighed trade offs or a misunderstanding about who is empowered to make a decision. It turns out these things are sometimes unclear.

It’s important to take a step back in moments like these to get a broader view of the situation: try to understand the position the other person is in, their priorities, how they may be weighing trade offs differently from you, whether they have context you don’t, and who should make the decision. Afterward, think about how to move the conversation forward in a way that’s constructive in the short and long term. Be direct, but diplomatic and try to address their concerns directly.

I understand using light gray for the text color here is important to you because it matches our logo, but it’s really important that the contrast is high so people with colorblindness can read it.

This example is intentionally low key — let’s set that aside. We start by showing empathy for our colleague’s point of view. We tell them we understand their concern and attempt to repeat it. This gives them a chance to correct us if there’s a misunderstanding on our part. We then share our proposed decision and the explicit trade offs we’ve made. In the first example above, we posited accessibility was more important than whether text color matches our logo color. Making implicit value judgements about trade offs explicit is key to building understanding.

If we don’t understand the concern then we ask clarifying questions.

I want to make sure I understand your concern. Are you worried about whether the text color matches the logo, or are you generally worried about brand guidelines? Is it something else?

Earlier I said to be “constructive in the short and long term” but what I really mean is if you treat people poorly then they will not want to work with you. If you do this repeatedly then you’ll have a bad reputation and, in turn, your effectiveness will go down over time as more teammates view working with you as a burden.

You might have been told to “disagree and commit” at some point in your career. Knowing when to fold ‘em, as they say (whoever “they” are), is an important tool in maintaining trust long term and maximizing your effectiveness at work. When deciding whether you should disagree and commit you should consider the long and short term ramifications of a decision[1] and whether you’re the person empowered to make the decision. For instance, if the decision is regarding something like the naming of an internal API then arguing until you get your way is going to erode some amount of trust from your teammate(s). Over time situations like this will cause you to be seen as an inflexible person. On the other hand, advocating for something like maintaining or increasing user privacy is likely really important and is worth fighting for.

You might also be asked to consult on a decision. This might come in the form of a friend or colleague asking for advice, or some sort of group setting that’s more formal. It’s important to recognize your place in these situations. Are you being asked to give input or are you the decision maker? It’s important to set your expectations accordingly to avoid disappointment if the designated decision maker makes a decision in conflict with your advice. If you don’t understand the decision then ask what trade offs they made or what additional context and inputs they took into account. Remember that they may weigh trade offs differently from you and that’s okay.

If you’re a decision maker asking others for advice, then set their expectations early. Some people may need you to tell them their input is valuable and appreciated, but you are also taking many inputs into account in some broader scope. These people may also need to be told directly that you may make a decision counter to their recommendation if other inputs need to be weighed more heavily. This approach can help avoid needless escalation and conflict.

Resolving conflict is more difficult if there’s not a well-defined decision maker. For example, consider the case where you and a peer have conflicting viewpoints on a decision and you’re at a stalemate after going through the process. In cases like this it’s helpful to bring in a third party to mediate or break the tie. It might also be helpful to consider whether the decision is reversible. If it is, then it’s easier to disagree and commit to move forward. If it doesn’t work out the team can take its lumps and change course. Most decisions are reversible.

A final note: it’s sometimes quite hard to strike the right balance between directness and respectful tone in text. Try talking to your counterpart face to face or video chat if a conversation goes sideways via text chat. It may remind you both there’s a human on the other side of the discussion.

[1] It’s important to keep ethics in mind here. If you’re fighting to uphold some ethical boundary then it’s always worth continuing to advocate for your position.

Finding Dependency Cycles in iOS Builds with llbuild UI

2018-04-03T08:14:28-07:00

Did you know the swift-llbuild project contains a debugging tool called llbuild-ui that can find dependency cycles in your iOS build? To try it:

Enable the New Build System (In Xcode it’s under File, Workspace Settings)
Build your project
Open the Derived Data directory on your machine and find the build.db file generated by the build system. This will be in a path similar to /Users/<yourUsername>/Library/Developer/Xcode/DerivedData/iOS-<someIdentifier>/Build/Intermediates.noindex/XCBuildData/build.db. Take note of this path as you will need it later.
Follow the instructions here to install llbuild UI. You’ll need to have Python (and probably some dependencies) installed on your system.
Start llbuild UI by running the command shown on the Github page linked in step 4 above in terminal.

FLASK_APP=llbuildui.app venv/bin/python -m flask run
Visit http://127.0.0.1:5000/ in your browser. You should see something like this:
Click on Database Browser, which will show you the database configuration screen:
Paste the path from step 2 above into the field and click “Set Path”
Wait until the page refreshes then change the address in your browser’s address bar to http://127.0.0.1:5000/db/diagnostics. Any cycles found in your build should be shown on screen.

Diagnosing "Dependency Cycle Between Targets" in Xcode 9's New Build System

2018-03-09T22:15:17-08:00

Xcode 9 includes a new build system that can substantially improve build times. The new build system is more strict about build issues, but some of its diagnostic output can be difficult to reason about. One difficult to debug error is “Dependency cycle between targets,” which may appear during incremental builds where you may not have an explicit circular build dependency between targets. Let’s talk about how these types of cycles are introduced and how to troubleshoot and fix them.

There is more than one way to cause this sort of target dependency cycle without adding an explicit target dependency. The behavior of the legacy build system is to continue to (re)build instead of failing. This can increase incremental build times for your project, but the build ultimately succeeds and you may not know there’s a problem. The new build system fails instead, which helps you better maintain your project and means better incremental build times once you’ve fixed the dependency issues.

During a full (clean) build the compiler will emit a list of dependencies for each compilation unit in your project. On subsequent (incremental builds), this data is used by the build system to help determine what parts of your project need to be rebuilt depending on what has changed since the last build. This list may include implicit dependencies (not to be confused with the per-scheme setting!) picked up by the compiler.

First, set this flag to get a bit more diagnostic info in the build log:

defaults write com.apple.dt.XCBuild EnableDebugActivityLogs YES

Reproduce the dependency cycle error and expand the related error message in the build log. Usually there’s a file name in the log. Open that file and make sure its imports are correct. For instance, if a library B is consumed by library A and library B also #imports a file in library A then there’s a circular dependency. Remove the offending import and build again.

There’s at least one more way to introduce implicit dependency cycles. For instance, modulemaps within header search paths will be added as dependencies for a given file automatically. In fact, a file may introduce a dependency cycle with its own module if the (e.g. static library manually specifying a modulemap) target containing the file has a modulemap in its Headers build phase and that phase is placed after the Compile Sources build phase. This can be resolved by dragging the Headers phase above the Compile Sources phase in the Build Phases subtab of your target’s settings in Xcode.

If this doesn’t work then it’s time to dig a bit deeper. You can see a list of dependencies (found by the compiler) for a given file by inspecting the contents of <filename>.d in ~/Library/Developer/Xcode/DerivedData/<project-name>-<some-identifier>/Build/Intermediates.noindex/<name>.build/Debug-<target-device>/<name>.build/Objects-normal/<arch>/ after performing a build.

Look for modulemaps that don’t belong in the dependency list. For instance, if the file is in library A and library A’s target doesn’t depend on libraries B and C, but B and C’s modulemaps appear in the dependency list then library A’s header search paths are likely misconfigured. Clean up library A’s the header search paths and try again.

This is probably not an exhaustive list of ways to cause and fix this error. You can let me know if you know more about this issue on Twitter.

Mock Interviews for Fun and Someone Else's Profit

2018-01-12T08:16:47-08:00

Last year I started offering mock interviews for iOS developers. It’s been a learning experience for me and, I think, others. I’d like to share a bit about what me and the 10 folks I’ve interviewed so far have experienced and what’s next.

Through the wonder of the internet, I’ve met (and hopefully helped) folks from all over the world. Once a week I interview an iOS developer from anywhere in the world during my lunch hour. I’ve interviewed people from the US, Canada, Italy, Australia, and more. I currently have another 8 interviews lined up with folks from Greece, Singapore, and Brazil to name a few.

Interviews start with about 15 minutes of iOS knowledge questions that cover a wide range of categories from user interface basics to concurrency. I tell candidates that real interviews will go much deeper on each of these topics. This helps them figure out the sort of things they may be asked in a real interview. Of course, I also explain that interviews vary from company to company. I don’t ask questions about ping-pong balls in boats or busses.

The knowledge portion of the interview is followed up by a brief programming exercise using a shared text editor. I say up front that I don’t care about syntax, but I do want to know whether candidates can solve the exercise and explain their rationale as they go. I explain I don’t care whether the code compiles or if they can remember the name of a specific API since tools help with that sort of thing in a typical setting. I do look for whether they find ways to improve their code or the skeleton API I provide. The exercise doesn’t revolve around a Computer Science brain-teaser or some clever trick, though you ought to know a common data structure or two.

Candid feedback is given as we go and at the end of the interview. I explain what I expect from folks starting out, mid-level, or senior folks. And I leave a few minutes at the end of the interview for candidates to ask questions. Of course, I say “I don’t know” when I can’t answer them.

Feedback from candidates has been positive and most of the interviews went rather well. Interviewing can be nerve-wracking so it’s rather gratifying to hear I’ve helped someone feel more comfortable with interviewing. I’ll keep doing this for at least the next few months.

P.S. Let me know on Twitter if you’re experienced and interested in giving mock interviews. I can share my topics and exercise if you’re an iOS developer. Otherwise, you’ll need to put together a plan of your own.

Getting Started with Home Automation

2017-05-19T17:43:25-07:00

Home automation is still an early adopter’s game. The landscape is fragmented and there are competing standards like Insteon, Z-Wave, zigbee, Wi-Fi based devices like Nest and Philips Hue, and Apple HomeKit. And most of these things don’t work together out of the box. ?

So what should you buy? Will it work with iOS via HomeKit or do you have to use a bunch of apps?

I have experience with Insteon, Z-Wave, and Apple HomeKit so I’ll share what I know about those standards here. But first, let’s get some basics out of the way. Setting up Home Automation requires a controller and a responder. (I’ll call responders devices here.)

Controllers
There are many controllers out there. First party controllers generally work with one standard. For instance, Insteon’s hub only talks to Insteon devices. And Samsung’s SmartThings hub talks to Z-Wave devices. I should point out that Z-Wave is not Samsung’s standard. It’s a standard released out by an “alliance” of companies.

You’ll find many Insteon controllers on the market. Insteon makes an “Insteon Hub” (Amazon link), for starters, which is an inexpensive way to jump in. You can use Insteon’s app to control stuff around your house and on its own this is okay. Confusingly, there are two versions of the Insteon Hub. One compatible with Apple HomeKit and one compatible with basically everything else, including Amazon Alexa-based devices like the Echo. My understanding is this is due to HomeKit’s licensing terms. This is not great since it introduces additional fragmentation into the home automation ecosystem (Boo Apple). On the other hand, it probably enforces some minimum level of security, which Apple cares deeply about (Yay Apple). We’ll talk a little more about HomeKit and a way to work around this limitation later.

There are also third party controllers that support more than one standard. The most popular controller for “hard core” home automation enthusiasts is the ISY994. It’s a small appliance-like device. The configuration interface is horrible, imo, but it’s the most flexible controller out there and it’s quite reliable, from what I understand. The ISY supports Insteon, Z-Wave, and infrared control though you may need to pay a fee to unlock additional functionality depending on what version you buy. Note that you’ll need to buy an Insteon modem to allow the ISY communicate with your Insteon devices. The Z-Wave radio is built in if you buy a model that supports Z-Wave. If I were starting from scratch today, I’d probably go with an ISY994 controller.

But I’m not starting from scratch today. I use a software controller called Indigo. Indigo is fantastic, native software for macOS that is flexible and has a familiar user interface. While I wouldn’t call setting complicated flows in Indigo “easy,” I would say it’s about as easy as it gets for a fully-featured home automation controller. It has an active user base and plugin ecosystem. For instance, I once used a weather plugin to control whether or not my irrigation system came on outside my home, in addition to a normal schedule, to save water. If the forecast called for rain then my sprinklers would not turn on. We’ve since moved to a home without an irrigation system so I don’t need this anymore. Indigo’s license cost is an annual subscription. It’s a reasonable cost and well worth it to me. Indigo does need to run on a Mac all the time. I run it on an old Mac Mini that serves as my Time Machine Server and more.

Perceptive Automation, the company that makes Indigo, updates the software with reasonable regularity and runs a service, called Reflector, that lets their iOS app access your Indigo installation externally securely and transparently. Their latest major release added support for Z-Wave door locks and more. You’ll need to buy USB “modems” to communicate with your devices (Insteon, Z-Wave).

Many controllers expose some sort of HTTP API. That’s the case for the ISY and Indigo. This is will come into play later when we try to use HomeKit with these things.

I don’t have any experience with Samsung’s SmartThings, but it seems to be well-liked and have broad hardware support.

There are many other controllers out there, like the Vera and the now defunct Revolv (?). The latter aimed to support multiple standards. The software was great but the company was purchased and the service was ultimately shut down. You can look into the Vera more, if you like.

HomeKit works a bit differently. Each device you buy can be its own controller or require a separate controller. It varies. You pair the controller with your iOS device and it’s exposed on your phone via Apple’s Home app or third party apps that implement HomeKit. iOS syncs your HomeKit database and settings via iCloud. It works quite well it is proprietary as one might expect from Apple. You can setup automation flows if you have a fourth generation Apple TV in your home. For instance, I use HomeKit automation to turn on the lights on my main floor when I get home. It was incredibly easy to set up. I do this despite having no support HomeKit devices thanks to an open source project called Homebridge. We’ll talk more about that below.

Insteon
Insteon makes a variety of devices like lamp controllers, switches, dimmers, keypads, fan controllers, irrigation controllers, garage door controllers, and various sensors. Insteon gear can be configured with or without a central controller and works even if your controller is down or is unavailable for some reason.

Insteon devices communicate wirelessly or via the neutral wire network in your house. Some devices are wireless only (e.g. battery powered sensors). Insteon devices relay commands to each other via a mesh network.

Third party Insteon device selection is limited.

I use mostly Insteon devices and they work great, with the exception of Insteon fan controllers which seem to fail after some time in my experience. I’ve had three fan controllers fail in a way where one of the fan speeds did not work properly. Insteon’s has great support so I don’t sweat this too much. Also, the fan controller’s speeds are a little odd. Full speed works as expected, medium is a bit too slow (maybe 30-40% instead of 50%), and low is way too slow.

I purchase my Insteon gear from smarthome.com when they run sales. This usually happens around major US holidays like the Fourth of July. Otherwise, I buy from Amazon because their prices are a little better.

Z-Wave
As I mentioned above, Z-Wave is a standard maintained by several companies. Z-Wave devices come in a lot of shapes and sizes and are made by many manufacturers so you can almost always find what you’re looking for. Here’s an unobtrusive, expensive, and amazing Z-Wave door/window sensor, for instance. The range on this sensor isn’t great, but a cheap Z-Wave repeater fixes that. I should mention that any plugged in Z-Wave device can act as a repeater. It just turns out that I don’t have many Z-Wave devices in my home so range was limited. This would not be an issue for most setups.

I have less first-hand experience with Z-Wave. I recently purchased and installed a Z-Wave deadbolt and the sensor and repeater I mentioned above. They were all quite easy to get working in Indigo, which picked them up automatically.

The only confusing thing was getting the repeater to work properly with the door sensor. I had to “optimize the Z-Wave network.” Presumably Indigo looked at the devices and gauged signal strength to find the optimal routes in the network then updated the settings of each to reflect those routes. This isn’t something I have ever had to do with my Insteon devices – routing just plain works.

I would probably purchase more Z-Wave devices today if I were starting from scratch because there is a variety of hardware to choose from. But it’s worth noting that Z-Wave devices seem to be slightly harder to set up [in my limited experience].

Wi-Fi Based Stuff
Chances are you’ve already setup a robust Wi-Fi network in your home. Many HomeKit devices and other devices like Nest thermostats use W-Fi. I’m not a fan of this because stuff stops working when I reboot my router. It’s an edge case, admittedly, but it’s one that matters to me. Nest devices are, unsurprisingly, not HomeKit compatible. But I already use Nest thermostats in my home. There are other nice thermostats out there but Nest thermostats look the best and work really well.

Homebridge
iOS 10 added useful (and solid!) first party HomeKit support via the Home app and a favorites pane in Control Center. It also enables Siri integration. But the majority of home automation devices available today don’t support HomeKit.

Homebridge to the rescue. Homebridge is an open source project that exposes your home automation controller’s API as a HomeKit device using plugins.

Practically this means you can use non-HomeKit devices with HomeKit. Insteon, Z-Wave, Nest, and lots more devices can work with your iOS device mostly pain free provided there’s a plugin available or you’re willing to write one yourself (in Javascript). You do need to run Homebridge on a server in your home. I run this on my Mac Mini but many people run it on a Raspberry Pi. Homebridge takes a little understanding to set up. Programmers and technically inclined people will feel at home. It basically just works once it’s running, though I have had the occasional crash in a plugin cause Homebridge to crash. I run an alias to start it again automatically if that happens:

alias start_homebridge=‘while true; do homebridge; sleep 1; echo “Restarting Homebridge”; done;’

I use Homebridge to expose Insteon and Z-Wave devices via Indigo as well as my Nest thermostats to iOS. It gives me the best of both worlds: the flexibility to choose from a wide variety of hardware and the ability to control parts of my home with my iOS devices. Apple could break this support in the future (intentionally or not) and note that I’ve not been able to get the iOS’ HomeKit widgets to work with Homebridge. I don’t know why that is.

hey siri, turn off the fan in the office.

There’s a lot about home automation this post doesn’t cover. I’m happy to answer your questions, as time allows, on Twitter though my answer might be “I don’t know!”

I should mention that my good friend Drew got me into this hobby a few years back. Much of what I shared above is information I picked up from him over the years along with what I’ve found though lots and lots of reading. Thanks for getting me hooked, Drew! ???