Blake Niemyjski

Agentic Driven Development (ADD): AGENTS.md, Skills, and the Full Workflow

Blake Niemyjski — Wed, 04 Mar 2026 00:00:00 GMT

I've been spending a lot of time over the past year integrating AI agents into my development workflow across several projects. It started with .github/copilot-instructions.md, which was GitHub Copilot's first attempt at letting you give the AI project context. That was a good start, but it was a single monolithic file tied to one tool. Then AGENTS.md came along as an agent-agnostic standard, and more recently agent skills have taken things even further with domain-specific instruction files that get loaded on demand. The progression from "one big instruction file for one tool" to "a curated set of skills that any agent can use" has been really interesting to watch and build on.

I wanted to share what I've learned across that journey, what's working, and where I think this needs to go.

What Are AGENTS.md and Agent Skills?

AGENTS.md is an open format for giving coding agents project-specific context. It emerged from collaboration across the AI coding ecosystem and is now stewarded by the Agentic AI Foundation under the Linux Foundation. It's a markdown file you place at the root of your repository that tells agents how to build, test, and navigate the codebase, what conventions to follow, what architectural decisions matter, and what pitfalls to avoid. Think of it as onboarding documentation, but for AI.

It's already been adopted by over 60,000 open source projects, and the list of tools that support it keeps growing: VS Code (v1.105+), GitHub Copilot, Cursor, Windsurf, Gemini CLI, Zed, Warp, Aider, and RooCode. Claude Code is one notable exception here. It uses CLAUDE.md instead of AGENTS.md, which I'll come back to later.

Agent skills take this further. They're focused instruction files (SKILL.md) organized into directories like .agents/skills/, .github/skills/, or .claude/skills/ depending on the tool, and they give the AI deep domain knowledge about specific areas of your project. When the agent is working on frontend code, it loads the frontend skill. When it's writing tests, it loads the testing skill.

What I really like about this is that a lot of this structure works across multiple agents. Support and invocation behavior still vary by tool, but you can increasingly write these instructions once and benefit across more than one agent, which is a big step away from vendor lock-in.

One thing to know: skills don't always get invoked when you'd expect. Sometimes the agent will just announce that a skill exists instead of actually using it, or it'll ignore a relevant skill entirely. In VS Code, I have had better results with the experimental chat.experimental.useSkillAdherencePrompt setting. It uses a stronger adherence prompt that pushes the model to invoke relevant skills instead of just mentioning them. I've found this makes a noticeable difference, but since it's experimental, I'd still treat it as something to test in your own workflow rather than assume across every setup.

Skip the Hierarchy, Keep It Flat

Some tools support hierarchical AGENTS.md files: parent directories, subdirectories, nested overrides. VS Code even has a chat.useNestedAgentsMdFiles setting for it. In theory this lets you share common conventions at the top and specialize deeper in the tree. In practice, I've had bad luck with it. Agents don't always pick up the right files in the right order, rules from different levels can conflict in confusing ways, and debugging which instructions actually got loaded is a pain.

My recommendation: use a single AGENTS.md at the repo root and put everything else in skills. The root file stays concise: build commands, project structure, high-level conventions, and a table pointing to your skills. Domain-specific knowledge goes in skill files that get loaded on demand. This is simpler to reason about, easier to maintain, and more predictable across different agents.

How We Use This in Practice

Foundatio

Foundatio is an open source library providing pluggable building blocks for distributed .NET applications (caching, queues, locks, messaging, jobs, file storage). The AGENTS.md file covers:

Repository overview, what each abstraction does and the design principles (interface-first, testable, swappable implementations)
Quick start commands, dotnet build, dotnet test, dotnet format
Project structure, mapping out src/, tests/, samples/, benchmarks/
Coding standards, from style rules to architecture patterns like async suffixes, cancellation token conventions, and when to use ValueTask<T>
Common patterns, extension methods, logging, exception types, single responsibility guidance

We also use the AGENTS.md to keep the documentation up to date. When the agent makes changes to Foundatio, the instructions tell it to update the relevant docs as part of the same task. This means the README and API docs stay in sync with the code without us having to remember to do it manually.

The key insight here is the "Craftsmanship Mindset" preamble: "Every line of code should be intentional, readable, and maintainable. Write code you'd be proud to have reviewed by senior engineers." Setting the tone matters. The agent will mirror the standards you set.

This has been a really important pattern in my own prompting: the opening of your instructions sets the tone for everything that follows. If you lead with "write clean, production-ready code," the model approaches the task differently than if you lead with a list of file paths. I wouldn't treat this as a universal law for every model and harness, but in practice I've found that putting your standards right at the top of AGENTS.md noticeably improves the quality bar.

Some official model documentation suggests this matters less than it used to. I've seen it firsthand though, and in my workflow it makes a massive difference when working with system prompts. If you want to go deeper on prompting techniques, I'd recommend reading the Anthropic prompting guide, the OpenAI prompt engineering guide, the Codex prompting guide, and the Google Gemini prompting strategies. Each provider has slightly different recommendations, but they all agree that clarity and structure matter.

It's also worth noting that some models and harnesses actually work better with less in the AGENTS.md. Codex is a good example of this, where a leaner, more focused context file can outperform a comprehensive one. The arxiv study on evaluating AGENTS.md found exactly this: unnecessary requirements made tasks harder for agents, not easier. So know your tools and test what works. More isn't always better.

Exceptionless

Exceptionless is a real-time error monitoring platform (ASP.NET Core 10 + Svelte 5) and this is where we went much deeper with skills. The AGENTS.md is relatively concise, pointing agents to load specific skills from .agents/skills/<name>/SKILL.md based on the domain they're working in:

Domain	Skills
Backend	dotnet-conventions, backend-architecture, dotnet-cli, backend-testing, foundatio
Frontend	svelte-components, tanstack-form, tanstack-query, shadcn-svelte, typescript-conventions, frontend-architecture, storybook, accessibility, frontend-design
Testing	frontend-testing, e2e-testing
Cross-cutting	security-principles, releasenotes
Billing	stripe-best-practices, upgrade-stripe
Agents	agent-browser, dogfood

That's 22 skills total, including 6 third-party ones (browser automation, dogfood, frontend design, release notes, Stripe) that we manage separately. When an agent is working on a Svelte component, it knows to use shadcn-svelte patterns, TanStack Query for data fetching, and our specific component conventions. When it's working on the backend, it understands our repository patterns, Foundatio abstractions, and how we use FluentValidation.

Third-party skills are managed with the Vercel Skills CLI. You add them with npx skills add and keep them up to date with npx skills update, which uses a lock file to track versions. It's really clean. Companies like Stripe and Anthropic publish official skills for their products, so instead of writing your own Stripe integration guide, you just pull theirs. There's also skills.sh, a directory for discovering community and official skills across platforms.

Vercel has done a really great job with the npx skills CLI. It's the best tool out there right now for managing third-party skills. That said, it feels a bit odd running npx and pulling in Node.js as a dependency for a .NET-only project that doesn't otherwise have a package.json. You end up with a Node toolchain just to manage your agent skill files, which is a weird ask for a team that's all-in on dotnet. I'd really love to see a dotnet skills tool or even a dotnet tool install skills that does the same thing natively. If anyone's looking for an open source project to start, there's your idea!

Security Risks of Third-Party Skills

One thing I want to be really clear about: third-party skills are code that directly influences what your AI agent does. They're instructions that get injected into the agent's context, and a malicious skill could tell the agent to do things you really don't want, like exfiltrating secrets, introducing backdoors, or modifying files outside the scope of the task. This is a real supply chain risk, similar to pulling in an untrusted npm package or NuGet dependency.

Here's how I handle it:

Always commit skills to your repo. Don't pull them at build time or on the fly. They should be checked into source control so you have a clear record of what's in your project.
Review skill updates like code changes. When you run npx skills add to update a third-party skill, treat the diff the same way you'd treat a dependency update. Read what changed. Make sure nothing sketchy got introduced. Don't just blindly accept updates.
Run independent security scans. Tools like CodeQL and Copilot code review can catch some of this, but you should also be scanning your skill files as part of your security pipeline. If a skill file suddenly tells the agent to ignore security conventions or access environment variables, that's a red flag. skills.sh does a nice job here by having three different vendors scan each skill in their registry. That's a good start, but you still need to review every single change yourself. At the end of the day, skills are instruction-level supply chain risk. Treat them with the same seriousness you would any other dependency that can influence execution.
Prefer official skills from known sources. Skills published by Stripe, Anthropic, Vercel, and Microsoft carry a lot more trust than random community skills. That doesn't mean community skills are bad, but you should be more careful vetting them.

This is still a pretty new ecosystem, so the tooling around skill verification and trust is still evolving. But the principles are the same as any other supply chain security: know what you're pulling in, review it before you trust it, and keep a record of changes.

This Blog

Even this Astro blog has skills. There's a project-structure skill documenting the site layout, a brand-guidelines skill for visual identity and voice, and an seo-schema-markup skill for JSON-LD structured data. When I ask an agent to write a blog post, it knows the frontmatter schema, the content directory structure, and the tech stack (Astro 5, Tailwind CSS v4, Pagefind).

Self-Learning: The Promise and the Trap

Exceptionless has a "Continuous Improvement" section in its AGENTS.md:

Each time you complete a task or learn important information about the project, you must update the AGENTS.md, README.md, or relevant skill files.

This is the self-learning pattern. The agent documents what it discovers as it works. In theory, the knowledge base grows organically over time.

In practice, this needs guardrails. Without them, you end up with bloated instruction files full of edge cases, one-off fixes, and stale information. It becomes like a FAQ that nobody maintains, every entry technically true but not really helpful because the useful stuff gets buried.

What I think works better is treating skills more like curated knowledge than an append-only log. A few principles:

Be selective about what gets documented. Not every discovery is worth persisting. A quirky workaround for a dependency bug isn't a "skill", it's a footnote that'll be irrelevant after the next update.
New features should trigger pruning. When you add a new capability or refactor a system, that's your cue to review the relevant skill files. Remove rules that reference old patterns. Update conventions that have evolved. Skills should reflect the codebase as it is, not as it was.
Guard against tech debt in your instructions. Ironically, the files meant to prevent tech debt can become tech debt themselves. If a skill file has contradictory rules, outdated examples, or references to deleted code, the agent gets confused and produces worse output. Maintain your instruction files with the same discipline you maintain your code.
Think FAQ, not encyclopedia. Document the patterns and decisions that come up repeatedly. If you've explained the same architectural choice three times in code reviews, that's a skill entry. If it's a one-off edge case, let it go.

There's actually academic research that backs up the "less is more" approach. A recent study, "Evaluating AGENTS.md", found that context files can actually reduce agent success rates compared to providing no context at all, while increasing inference costs by over 20%. The key finding is that unnecessary requirements make tasks harder for agents. LLM-generated context files performed worst, while human-written files with minimal requirements did slightly better.

Interestingly, Vercel's own evals tell a different story. They found that embedding documentation directly in AGENTS.md dramatically outperformed skills, hitting 100% pass rates on build, lint, and test metrics while skills with default settings scored the same as having no docs at all (53%). The key difference is that AGENTS.md content is always in the system prompt, so the agent doesn't have to decide whether to look something up. Skills require the agent to make a decision to invoke them, and that decision point is where things break down. Even with explicit instructions to use skills, they only got to 79%.

So how do you reconcile those two findings? I think the arxiv paper is really about bloated, low-quality context hurting more than helping. And Vercel's results are about the right context, compressed aggressively (they went from 40KB to 8KB while keeping 100% accuracy), delivered passively. Both point to the same conclusion: less is more, and the best context is focused, curated, and always available.

It's also worth mentioning Context7 here. Context7 came before the skills ecosystem and is still really relevant. It's an MCP server that fetches up-to-date, version-specific library documentation directly into your prompt. So instead of baking framework docs into your AGENTS.md or skills (where they'll go stale), Context7 pulls the latest docs on demand. I use it alongside skills, where skills handle project-specific conventions and Context7 handles third-party library documentation. They complement each other well.

The takeaway from all of this is: keep your AGENTS.md focused on what actually matters, compress aggressively, and use tools like Context7 for documentation that changes frequently. Don't dump everything you know into one file. The agents in the arxiv study did follow the instructions and explore more thoroughly, but all that extra exploration didn't translate to better results when the instructions were bloated.

MCP Servers: Powerful but Painful

I have to talk about Model Context Protocol (MCP) servers here because they're becoming a really important part of the agent ecosystem, but the experience of using them is still pretty rough.

MCP servers let you give agents access to external tools and data sources: databases, APIs, browser automation, file systems, documentation fetchers like Context7, and more. The idea is great. Instead of the agent only being able to read and write files, it can interact with the outside world through a standardized protocol. In practice though, there are a few real pain points.

Context bloat just from being enabled. Every MCP server you have enabled adds its tool definitions to the agent's context, even if you never use it in that session. If you've got ten MCP servers configured, that's a lot of token budget being spent on tool descriptions before you've even asked a question. This ties back to the "less is more" theme. Only enable the MCP servers you actually need for the project you're working on, and disable the rest.

Configuration is different everywhere. Every tool has its own way of setting up MCP servers. Claude Code uses a claude_desktop_config.json or .mcp.json, VS Code has its own config format, Cursor does it differently, and so on. If you're using multiple tools (and you should be, as I talked about with the cross-model review workflow), you end up maintaining the same MCP server configuration in three or four different places. There's no shared standard for this yet.

Discovery is inconsistent across tools. Some tools are getting better at this. Cursor has an MCP directory built right in, and VS Code has one too. But not all tools have caught up, and there's no universal npx skills add equivalent for MCPs. Depending on which tool you're using, you might have a nice browsable directory or you might be searching GitHub and hoping for the best. Compare that to the skills ecosystem where skills.sh gives you a searchable directory and the Vercel CLI handles installation regardless of which agent you're using.

Security is a huge concern. This is the one that really worries me. MCP servers can execute arbitrary code, make network requests, access your file system, and interact with external services. A malicious or compromised MCP server has way more attack surface than a malicious skill file. A skill file can only influence what the agent thinks it should do. An MCP server can actually do things on your machine. What's worse, some editors like VS Code ship MCP tools that haven't been through a thorough security review and are enabled by default. That's a lot of trust to place in tooling that most developers haven't audited.

Here's what I recommend:

Only use MCP servers from trusted sources. This is even more important than with skills. Check the source code, check who maintains it, and check how actively it's maintained.
Audit the permissions. What can this server access? Does it need network access? File system access? Does it run with your user's full permissions? The less access it needs, the better.
Pin versions. Don't pull latest for MCP servers. Pin to a specific commit or version so you know exactly what you're running. Review changes before updating.
Watch for prompt injection. MCP servers return data that gets fed into the agent's context. If an MCP server returns data from an untrusted source (like user input from a database or content from the web), that data could contain prompt injection attacks that hijack the agent's behavior.

The MCP ecosystem is moving fast and the protocol itself is really well designed. But the tooling around discovery, cross-tool configuration, and security still has a long way to go. I'm hopeful this gets better quickly because MCPs are genuinely powerful when they work.

Planning Mode: Think Before You Code

One thing I've changed in my workflow is that I almost always run in planning mode. In Claude Code you can hit Shift+Tab to toggle into plan mode, or just type /plan. This puts the agent into a read-only or planning-first exploration mode where it analyzes your codebase and proposes a plan before touching any files.

Why does this matter? Without planning mode, agents tend to just start writing code immediately. They'll make assumptions about your architecture, pick the wrong file to modify, or go down a path that doesn't fit your patterns. With planning mode, the agent explores first, reads the relevant files, understands the structure, asks clarifying questions, and then presents a plan for your approval before it writes a single line.

This is especially valuable when you're working with skills. The agent loads the relevant skills during the planning phase and you can verify it's actually applying them before it starts coding. I've caught plenty of cases where the agent's initial approach would have conflicted with our conventions, and a quick redirect during planning saved a lot of back-and-forth.

I have to say, Cursor has the best planning experience I've used. I always get really great results from its planning mode. It's worth trying if you haven't already. The Claude CLI and GitHub Copilot CLI are also solid options for terminal-based planning and coding. One thing to note: Claude's tooling centers CLAUDE.md rather than AGENTS.md, so keep that in mind if you're working across multiple tools.

Steering with voice is something I've been really enjoying. VS Code 1.109 added message steering and queueing as an experimental feature, and with the VS Code Speech extension you can use voice to kick off a task, bounce ideas around, or learn new concepts by talking through them with an agent. It's a really natural interaction model and it's how I like to start my planning sessions now.

I also really like how Google Antigravity handles planning. It generates a structured plan and then lets you leave comments directly on it, similar to commenting on a Google Doc. The agent incorporates your feedback without stopping its execution flow. That's a really nice interaction model. I wish all the tools would adopt something like that. And honestly, all of them would be better if you could print or share the plan more easily.

Plans as Architectural Documents

One thing I've started thinking about differently is that plans aren't just throwaway artifacts. They're architectural documents. A good plan captures the "why" behind a set of changes: what problem you're solving, what approach you chose, what alternatives you considered. That's incredibly valuable context that usually gets lost once the code is written.

I've started saving plans alongside the code, either in a docs/ directory or in the PR description. Six months from now when someone asks "why did we do it this way?" that plan is the answer. You can also keep plans outside the codebase entirely, in a wiki or shared docs system, especially for bigger architectural decisions that span multiple repos. The point is: don't let that context disappear after the agent finishes coding.

This idea of "docs as the source of truth" extends beyond plans. Ralph is a really interesting project that takes PRDs (Product Requirements Documents) and turns them into the driving force of an autonomous agent loop. You write a PRD as a markdown doc, convert it into structured user stories, and Ralph picks up incomplete stories one at a time, implements them, runs quality checks, commits, and moves on. The PRD becomes the living spec that both humans and agents work from. I really like this pattern because it keeps the requirements front and center instead of buried in a ticket system. The PRD is the documentation, the acceptance criteria, and the progress tracker all in one.

OpenCode and Codex

I want to mention OpenCode here because it's a really solid alternative in the terminal agent space. It's open source, supports multiple LLM providers (75+ through Models.dev), and the TUI is really nice with multi-session support, LSP integration, and shareable session links.

I also really like Codex. For me it's one of the best experiences for agentic development right now. You kick off a task, iterate quickly, and focus on the end result rather than staring at every line of code. Some people call this "vibing," and honestly that's a pretty good description. The key is that you're not just blindly trusting the output. You're pairing that fast iteration with the multi-agent review workflow I described earlier, so you get the speed of vibing with the code quality of a proper review process. That combination is really powerful.

The reason I bring up all these tools is that it validates the agent-agnostic approach. If you invest time in good AGENTS.md and skill files, you're not tied to any one tool. You can switch between Claude Code, Codex, Copilot, OpenCode, and whatever comes next depending on the task, and they all benefit from the same project knowledge.

The Full Agent Workflow: PRs, Reviews, and Feedback Loops

Where this really clicks is when you wire it all together into a continuous feedback loop. Here's the workflow I've settled into:

1. Plan and Implement

Planning is the most important step. I use Claude or Codex for planning since they're really strong at architectural reasoning. Feed it all the context it needs: the ticket, acceptance criteria, relevant docs, architectural constraints. Let the agent explore the codebase, understand the architecture, and propose a plan. Then actually read the plan. Break it down step by step. Walk through the approach and iterate on it. Ask questions, push back on things that don't feel right, and have the agent revise until you're confident in the direction. This is where you catch bad approaches early, and the time you invest here saves you way more time later.

For implementation, I'll use Claude, Codex, or Gemini depending on the task. Gemini has been really solid for frontend work in particular. Then for code review, I always use a different model than whatever did the implementation. If Claude wrote it, Codex or Copilot reviews it. If Codex wrote it, Claude reviews it. The whole point is getting a different perspective.

2. Cross-Model Review Before Committing

Once the agent has an implementation, have sub-agents on different models review the changes. This is the multi-agent review I talked about earlier. Check the current branch, make sure everything looks right, and iterate until you're happy with the quality. The bottleneck in any agentic workflow is going to be at review time. This is where you need the most powerful (and expensive) models to catch what the cheaper ones miss.

It's really important not to let tech debt sneak in during this process. I always look at the diff while the agent is working, before every push, and always after. Don't just trust the agent and move on. Review the code as it's going in. If something looks off, stop and fix it now because it's way easier to address in the moment than after it's merged.

If I'm not sure about the implementation at this point, I always tell the agent to commit but don't push. This lets me iterate locally without anything going out to the team. I can review the diff, ask the agent to change things, commit again, and keep going until I'm confident. Only when I'm happy do I move to the next step.

3. Run Tests and E2E

Before you branch and push anything, have the agent run the full test suite. Unit tests, integration tests, whatever you've got. If anything fails, the agent fixes it and runs them again. Don't push broken code.

For frontend work, this is where the agent-browser skill is one of my favorites. It lets the agent do end-to-end browser testing, actually navigating your app, filling out forms, clicking buttons, and verifying the UI works as expected. It's a really nice way to catch issues that unit tests miss before anything goes out for review.

For .NET projects, the .NET Aspire MCP is a game changer here. It automatically launches all your resources (databases, caches, message brokers, everything), provides full logs for the entire stack, and makes running tests and agent code a breeze. Instead of manually spinning up dependencies or mocking everything, Aspire handles the orchestration so the agent can just run tests against a real environment.

4. Branch, Commit, and PR via GitHub CLI

I let the agent handle the entire git workflow using the GitHub CLI (gh). As defined in our AGENTS.md, the agent follows Git Flow, creates a feature or bug branch, commits with a structured message format like [feat] #123 Add retry logic to queue processor or [fix] #456 Handle null reference in queue, and opens a PR filling out the PR template. The agent knows how to do all of this. You don't need to hand it specific commands.

The agent writes better commit messages than I do half the time because it has the full diff context and the skill files telling it our conventions. No more "fix stuff" commits.

5. PR Review: Copilot, CodeQL, and Team

Once the PR is up, kick off the review process. GitHub Copilot automatically reviews it using your skill files and copilot-setup-steps.yml workflow. CodeQL runs static analysis for security and quality issues. And your team gets notified to review as well.

6. Address Review Comments

This is the part most people miss. Just tell the agent to go review the PR comments and CodeQL findings. It already knows how to use the GitHub CLI to pull that information. You don't need to hand it specific commands.

Ask the agent to review all the PR comments to determine which ones are valid. For valid issues, have it do a proper root cause analysis and fix the underlying problem, not just patch the symptom. Make sure it adds or updates test coverage for any fixes and that the overall code quality is solid. Then push the update, and have the agent reply to and resolve the comments on the PR.

7. The Review Loop

Now repeat. Copilot and CodeQL review the new changes. Your team looks at the updates. Pull any new comments back into the agent. Fix, push, reply, resolve. Keep running this loop until the feedback is clean.

For every PR, have the agent self-review the entire diff before the next round of human review. Sometimes stepping back and looking at the full picture catches things that incremental reviews miss. This should be a standard part of the loop, not just something you do for big changes.

8. The Compound Effect

This creates a real feedback loop:

Planning mode with a strong model ensures it thinks before coding
Cross-model review catches issues before you even push
Tests and E2E verify everything works before branching
GitHub CLI automates the branch, commit, and PR workflow
Copilot + CodeQL + team review against your conventions
Review comments feed back into the agent for RCA and fixes
Agent replies and resolves comments on the PR directly
Patterns that keep coming up get added to skills

The loop tightens over time. As your skills get better, the initial code gets better, the reviews find fewer issues, and the whole cycle speeds up.

You might ask, how is this process actually working? We've shipped some really technical features to the Foundatio codebase and apps with a very high quality bar to production using this workflow. Right now we're iterating on a massive rewrite of Foundatio.Parsers and Foundatio.Repositories for Elasticsearch 9, and all tests are currently passing. This isn't a toy workflow for side projects. It's how we're shipping real, complex infrastructure code.

Just remember: there's a bad actor out there who also has agents, and they never sleep. Put security processes into your workflow. Even if it's a one-liner that audits all changes for security issues and OWASP vulnerabilities, that's better than nothing. The agents working against you are getting better at the same rate as the ones working for you.

But the automated PR review pipeline is only half the story. The other half is what happens before you even push.

Where Developers Fit: Multi-Agent Review and Iteration

So if agents are writing more and more of the code, what are developers actually doing? I've been thinking about this a lot lately.

From everything I've seen, we're moving up the stack. Less time writing boilerplate, more time on the stuff that really matters: breaking down problems, reviewing architecture, defining acceptance criteria, and making sure the output is correct and secure. It's less "person who writes all the code" and more "person who orchestrates and validates the work." And honestly? I think that's a pretty exciting shift.

But I want to be clear: I'm not just kicking things off and walking away. I still look at the architecture and implementation closely. I question edge cases and flag problems I see. I still influence the design a ton, pushing back on complexity, asking about security implications, challenging whether an approach is really the right one. That human feedback loop is still really important. The agents handle a lot of the execution, but the critical thinking, the "wait, what about this scenario?" questions, that's still very much on you as the developer. At the end of the day, this is still code that you are responsible for and it has to ship at a very high quality bar.

Multi-Agent Review Before Human Review

Here's what I've been really excited about: instead of one agent producing code and immediately throwing it at a human reviewer, you have multiple agents review each other's work first. Think of it like having several reviewers with different perspectives do a first pass before the final review.

As I mentioned in the workflow above, I always use a different model for review than whatever did the implementation. They have different strengths and blind spots. A review from a different model catches things the original one missed, just like two developers with different backgrounds looking at the same code. Feed that feedback back to the implementing agent, let it address everything, then review again with the second model. Back and forth until it's clean. After a few rounds of this ping-pong, the code quality is really solid before a human ever touches it.

I've found that saying "review all changes in this branch" or "review this PR" helps a ton. Tell the agent to make sure every change was absolutely needed. Check for security issues, edge cases, readability, and performance problems. Being explicit about what you want reviewed makes the output way more useful. Tell the review agent to review as if the code needs world-class integration and is being deployed to production. Sounds simple, but it puts the review on a whole other level. The agent stops giving you surface-level "looks good" feedback and starts really digging into edge cases, error handling, and whether the code is actually robust enough. The bar you set in the prompt directly affects the quality of the review.

There are community projects formalizing this same concept. Anvil is a Copilot CLI plugin that does adversarial review across up to three different AI models (GPT, Gemini, Claude) that each try to find bugs, security holes, and logic errors. I haven't used it yet, but it's doing exactly what I've been doing manually and it's something I'm actively planning to look into. The disagreement between models is the feature. If one model thinks the code is fine but another flags a problem, that's exactly what you want caught before a human reviews it.

You can take this even further with specialized review passes: one agent focused on security auditing, another checking architectural patterns, another on code quality and naming conventions. Each one catches different things. By the time a human sits down to review, the obvious issues are already resolved. The human reviewer can focus on the high-level stuff: does this approach make sense? Does it fit the broader architecture? Are there edge cases the agents missed?

Breaking Down Acceptance Criteria Into Your Flow

One of the most underrated things you can do is feed your acceptance criteria directly into the agent workflow. Take your ticket or spec, break it into concrete acceptance criteria, and let the agent work through them one by one. The agent can even help you break down vague requirements into testable criteria, which is super useful when a ticket is a little hand-wavy.

Then each criterion becomes a checkpoint. Did the agent's implementation actually satisfy it? You can have a review agent validate each criterion against the code changes. This gives you a structured way to verify completeness instead of just eyeballing a diff and hoping nothing got missed.

Testing and Human QA

After the multi-agent review cycle, you hand off to testing agents. These can generate test cases from your acceptance criteria, write unit and integration tests, and even run them. If the tests don't pass, the agent loops back and fixes the issues before a human ever sees them.

If you don't have acceptance criteria yet, you can generate them quickly. Just describe what the feature should do and have the agent break it down into testable criteria. It's a really fast way to get structured coverage even when the spec is thin.

But here's the thing: you still need human QA. Agents are really good at verifying specific behaviors, but they don't have the intuition for "this feels wrong" or "this flow is confusing." Human QA catches the stuff that's technically correct but practically bad. The sweet spot is letting agents handle the mechanical verification so your QA team can focus on exploratory testing and user experience.

There are skills like Impeccable that are trying to help agents evaluate UI quality. It's not quite there yet in terms of "taste," but it's getting closer every day. I think this is one of the areas where we'll see a lot of improvement soon.

The next logical step here is agent skills specifically for QA. Imagine a skill that takes your acceptance criteria, builds automated tests from them (I really wish these lived in the repo alongside the code), and verifies regression scenarios. You could give higher priority to testing areas with changing requirements so the feedback cycle is even faster. This would let QA teams adapt to shifting specs without manually rewriting test plans every sprint.

Using Agents to Learn and Share Knowledge

One thing that doesn't get talked about enough is using agents as a learning tool. When I'm working on an unfamiliar part of the codebase, I'll ask the agent to explain how something works, walk me through the data flow, or diagram the architecture. It's like having a senior dev who's already read every file sitting next to you.

This is super useful for understanding the impact of a bug. Ask the agent to trace through the code and show you everywhere a change could have downstream effects. Have it do a proper root cause analysis and explain not just what went wrong but why. I've found that agents are really good at this kind of investigation because they can hold the entire codebase in context and follow the threads way faster than I can by reading files one at a time.

The knowledge sharing side is where it gets even more interesting. Ask the agent to generate mermaid diagrams of the architecture, data flows, or system interactions. These are incredibly useful for onboarding new team members, documenting complex systems, or explaining a bug fix in a PR description. Instead of spending an hour drawing a diagram by hand, the agent generates one in seconds and you can iterate on it.

I've started making this a regular part of my workflow. After finishing a complex feature or fixing a tricky bug, I'll ask the agent to write up a summary with diagrams explaining what changed and why. That documentation lives in the PR, in the docs, or in the skill files. It compounds over time and makes the whole team faster.

The Massive Leap in LLM Capabilities

I really cannot overstate how much these models have improved in just the past three to four months. The jump in code understanding, multi-file reasoning, and the ability to hold complex architectural context has been unlike anything I've seen before. Tasks that agents would consistently botch six months ago? They handle them reliably now. The multi-agent review pattern I described above wouldn't have been practical even a few months ago because the individual agents weren't reliable enough to trust their reviews.

I want to be honest though: none of this is perfect. Agents will hallucinate. They will lie to you with confidence. Code reviews from agents will miss things or flag false positives. But for everything they get wrong, they get a whole lot right. I've found some really impactful bugs, security issues, and architectural problems by following this process that I might have missed on my own. And it's only going to get better with time.

SDL and Code Scanning Are More Important Than Ever

This is something I want to really emphasize: with agents writing more code, your Security Development Lifecycle (SDL) and code scanning practices are more important than ever, not less. It's really easy for an LLM to sneak things in. Not maliciously, but because it's optimizing for "make the code work" and doesn't always think about security implications the way a human would. It might introduce a SQL injection vulnerability, hardcode a credential pattern, add an overly permissive CORS policy, or pull in a dependency with known CVEs, all while confidently telling you the code is correct.

This is why automated scanning needs to be a non-negotiable part of your pipeline:

Static analysis (SAST) with tools like CodeQL, Semgrep, or SonarQube should run on every PR. These catch the patterns that agents commonly introduce: injection vulnerabilities, insecure deserialization, hardcoded secrets, and more.
Dependency scanning with Dependabot, npm audit, or dotnet list package --vulnerable catches vulnerable packages the agent might add.
Secret scanning should be enabled at the repo level. Agents sometimes generate placeholder secrets or copy patterns from training data that look like real credentials.
Copilot code review adds another layer by reviewing against your skill files and conventions automatically on every PR.

The multi-agent review workflow I described helps here too. Having a second model review specifically for security issues catches a lot. But automated scanning is your safety net. Don't rely on any single layer. The whole point of defense in depth is that each layer catches what the others miss.

So I think now is the time to invest in your agent infrastructure. The AGENTS.md files, the skills, the review workflows, all of it gets more valuable as the models improve. You're not just building for today's capabilities. You're building the scaffolding that will compound as these models keep getting better. And based on the trajectory I've seen recently, they're going to keep getting better fast.

Hooks: The Next Big Development

I think hooks are where the next big leap in agent workflows is going to come from. Right now, tools like Claude Code, VS Code (preview), and Cursor support hooks that fire shell commands in response to events like tool calls, file edits, or commits. That's useful, but we're just scratching the surface.

Imagine hooks that automatically run code formatting after every file edit. Or hooks that pause the session to ask you a clarifying question before the agent goes down a path you didn't intend. Or hooks that kick off a sub-agent review, like Anvil's adversarial multi-model review, every time the agent finishes an implementation task. All of this while keeping the session alive so you don't lose context.

The hooks themselves already support all of this today. You can build any of these right now. The missing piece is that there's no prebuilt ecosystem for hooks like skills.sh is for skills. You have to write them yourself. Here are some examples of what you can do:

Auto-formatting on save. The agent writes code, a hook runs your formatter (Prettier, dotnet format, whatever) before the agent moves on. No more style nits in code review.
Security scanning. A hook that runs a security scanner after every file edit. Catch secrets being committed, detect known vulnerable patterns, or flag changes to authentication and authorization code before the agent moves on.
Dependency auditing. When the agent adds a new package, a hook runs npm audit or dotnet list package --vulnerable to check for known vulnerabilities before it gets committed.
Interactive checkpoints. Hooks that prompt the user for input at key decision points. "The agent wants to add a new dependency, do you approve?" This keeps you in the loop without you having to watch every step.
Automated sub-agent review. After the agent finishes a task, a hook automatically sends the changes to a review agent (or multiple review agents) before proceeding. This is basically Anvil's pattern built into the workflow as a hook.
Test-on-commit hooks. Before the agent commits, run the test suite. If tests fail, feed the failures back to the agent automatically. No human intervention needed for the fix cycle.
License compliance. A hook that checks any new dependencies against your approved license list. Block the agent from adding GPL code to your MIT project, for example.
Skill and docs updates. After a refactor, a hook could prompt the agent to review and update relevant skill files and documentation.

I'd love to see a community hooks registry similar to skills.sh emerge. Prebuilt, vetted hooks for common patterns like security scanning, formatting, and compliance would save everyone a lot of time.

GitHub is already moving in this direction with gh-aw (GitHub Agentic Workflows), which lets you write agentic workflows in natural language markdown and run them in GitHub Actions. It's read-only by default with sandboxed execution, input sanitization, and team-gating for critical operations. This is the kind of thing I'm talking about, taking the agent workflow and making it a first-class part of your CI/CD pipeline.

I also think we're going to see a lot more investment in sandboxing for agents. Right now, most agents run with your full user permissions, which is a pretty big trust surface. As agents get more autonomous and hooks let them do more without human intervention, proper sandboxing becomes critical. Containers, restricted file system access, network isolation, scoped API tokens. The tools that figure out how to give agents enough access to be useful while limiting the blast radius of mistakes (or compromises) are going to win.

The tools that get hooks and sandboxing right are going to have a huge advantage. Hooks are what turn an agent from a tool you interact with into a workflow you orchestrate. I'm really excited to see where this goes.

The CLAUDE.md Problem

If you're using Claude Code, you need to know that Anthropic has chosen not to support the AGENTS.md standard. Claude Code uses its own CLAUDE.md format instead. As of March 4, 2026, there's an open issue with nearly 3,000 upvotes requesting AGENTS.md support, but no official response yet. It's frustrating because a growing set of major tools support the standard while Claude still requires its own parallel file.

So if you want your agent instructions to work across Claude Code and everything else, you need to maintain both files. The easiest workaround is to just copy your AGENTS.md content into a CLAUDE.md file and keep them in sync. It's annoying, but it works. Some people use hard links (a way to make two filenames point to the same file on disk), but git treats them as separate files anyway, so you're still managing two copies in version control. I'm really hoping Anthropic just adds AGENTS.md support so we can stop dealing with this.

Migrating from Copilot Instructions

If you're still using .github/copilot-instructions.md, I'd recommend migrating to AGENTS.md and skills. Use the most advanced model you can (like Opus), feed it the documentation for AGENTS.md and agent skills, find existing skills or an AGENTS.md that's really close to what you're thinking or is specific to the tech stack you're using as a base, tweak it in plan mode, and go from there. Tell it to be very surgical. The migration is straightforward since it's all just markdown. The big advantage is that you're no longer locked into one vendor's instruction format. Copilot and a growing set of other tools support AGENTS.md, even though Claude Code still requires CLAUDE.md. And the skill-based approach scales much better than a single monolithic instruction file.

The Future: Orchestration by Powerful Models

Looking ahead, I think the biggest shift is going to be orchestration by a powerful language model. Instead of you manually kicking off agents, reviewing, sending to another agent, and managing the loop yourself, a top-tier model acts as the conductor. It breaks down the task, delegates to specialized sub-agents (planning, implementation, code review, testing, security), coordinates their work in parallel, and synthesizes the results. You describe what you want, and the orchestrator figures out how to get there.

We're already seeing this take shape. VS Code 1.109 introduced agent orchestration as a first-class concept, where multiple specialized agents collaborate on complex tasks. Each sub-agent operates in its own context window (solving the context overflow problem), can use different models optimized for its role, and independent tasks run in parallel. The community has already built orchestration systems on top of this like Copilot Orchestra and GitHub Copilot Atlas.

Distribution has always been the hardest part of any computing problem, and agent orchestration is no different. You're starting to see Google push this forward with Gemini, where the model itself handles the coordination across multiple agents and tools. I think this is the natural evolution of everything I've described in this post. The workflow I'm doing manually today (plan, implement, cross-model review, test, PR, review loop) is going to be something you describe once and an orchestrator runs end to end.

The key ingredient is that the orchestrating model needs to be really powerful. It needs to understand your codebase, your conventions (through AGENTS.md and skills), the strengths of different sub-agents, and when to escalate to a human. This is why investing in your agent infrastructure now matters so much. The orchestrators are coming, and the teams with the best AGENTS.md files, skills, and review patterns are going to get the most out of them.

The Cost Rug Pull

I do think there's going to be a rug pull on costs at some point. Right now, models are really cheap with incredible value. Codex, Copilot, and Gemini subscriptions let you do a ton for very little money. But be careful with the "newer is cheaper" narrative. I've read breakdowns of both Gemini and OpenAI where a newer model claims lower per-token pricing, but the model emits way more tokens to accomplish the same task, so the actual cost ends up being higher. Always look at total cost for a task, not just the per-token rate.

This is why open source models are so important. You can run models locally today, and the latest developments around running models via Apple's Neural Engine are really exciting. Projects like ANE have potential to bring real speed, efficiency, and cost savings for local inference. Having the option to run locally means you're not entirely dependent on subscription pricing that could change at any time.

What's Coming Fast

I think we're going to see some really exciting iteration speeds on things that used to be painful. Localization, accessibility, documentation. Every major product is going to ship with an MCP server or skill. We're already seeing it with Stripe, Vercel, and shadcn. That trend is only going to accelerate.

Relatedly, website traffic is also declining across the board. Look at what happened to Tailwind CSS and other documentation-heavy sites. A lot of this is because AI is being used for search now, and those models are being trained on your content. It's worth asking: does your website have an llms.txt? If not, you might want to think about how your content is being consumed in an AI-first world.

Just remember, it's okay not to be the first. Things are moving so fast right now. Just be aware of what's out there and implement when there are clear winners.

Future Tech Debt (or the Lack of It)

This one is a little further out, but I think we're going to get to a place where if you have enough tests, requirements, and acceptance criteria, you can just regenerate your application. Point it at your existing data stores, your API contracts, your test suites, and let the agent rebuild the implementation from scratch. New tech debt appears until it gets regenerated again. It sounds wild, but it's the logical conclusion of everything I've described in this post.

That said, it's still going to be really important for a person to put tools and processes in place (with agent assistance) to make sure APIs aren't broken and users are happy. But will apps even be apps as we know them? Or will they be more like personal assistants feeding you data through your glasses or via voice? I think this is where OpenAI is going with their voice-driven Jarvis product. Just remember: your privacy and security should always be paramount.

Getting Better Over Time

The real value compounds. Here's how I think about the improvement cycle:

Start small. An AGENTS.md with build commands, project structure, and your top 5 coding conventions.
Add skills as pain points emerge. When you notice the agent consistently getting something wrong, that's a skill entry. Don't preemptively document everything.
Review skills during refactors. Changing architecture? Update the skills. Dropping a dependency? Remove references to it. Skills are living documents.
Use code review feedback as input. If the same comment keeps coming up in PRs (from humans or agents), codify it in a skill.
Prune ruthlessly. A shorter, accurate skill file outperforms a longer, stale one every time. The agent has a context window. Respect it.

The goal isn't to document everything. It's to document the right things, the patterns and decisions that make your codebase yours, so that AI agents can be productive contributors rather than generic code generators.

One more thing worth keeping in mind: prompt placement and repetition matter more than people expect. I first saw this framed well in this tweet, and it stuck with me because it matches what I've seen in practice. This has real implications for how you structure your AGENTS.md. Your most critical conventions should be stated clearly and prominently, not buried in a wall of text. If the agent has to wade through dozens of rules to find the important ones, the important ones lose their impact.

If you're not using AGENTS.md yet, I'd really encourage you to try it. Start with build commands and your most important conventions. You'll be surprised how much better the AI output gets with even a little bit of context. Then try the cross-model review workflow: have one agent write, another review, and iterate until it's clean. The combination of good skills and multi-agent review has been a pretty huge improvement for the quality of code coming out of my projects.

Prompting is a skill worth investing in. A great prompt has three things: context (what you're working on, the tech stack, the constraints), intent (what you want the agent to do and why), and a definition of done (what a good result actually looks like). Be specific. "Refactor this function" is okay. "Refactor this function to use the repository pattern from our Foundatio skill, keep backward compatibility, and add unit tests" is way better. And if you're not getting good results, don't just ask the same thing louder. Come at it from a different angle. Try "tell me everything wrong with approach X" and then solve for it. Sometimes changing the prompt is like going for a walk and coming back with fresh eyes. A different framing can completely change the output.

And if you're already doing all of this, I'd love to hear how you're handling the maintenance and pruning side of things. That's where I think the community still has a lot to figure out.

For those just entering the field: there is a whole lot more to learn than just prompting, especially around security. You don't always have to be the first to do something, just execute well. Learn to prompt effectively, but also learn how these models actually work (it's next-word prediction, all the way down). Use AI as a tool. Trust that it will get you most of the way there, but know that it's always that last 10-20% that's the hardest. That's where your skills, your judgment, and your experience make all the difference.

One last thing: this blog was converted from Hugo to Astro in a few hours via OpenAI Codex. I didn't write a single line of code, just reviewed it and iterated. This blog post was written by Claude Code (Opus 4.6) with a skill built from my writing voice across all my prior blog posts, reviewed by the Codex app, and proofread and agentic-looped by yours truly. That's where we're at right now. Never stop learning, trust your gut, and ship it.

How-to get notifications when your mailbox is opened

Blake Niemyjski — Thu, 17 Dec 2020 00:00:00 GMT

Have you ever waited endlessly for an important package or mail to be delivered? Have you ever dealt with a package being stolen from your mailbox? In this guide I'll show you how to see packages will arrive today and notifications when it's delivered as shown below with the help of Home Assistant!

This guide will also show you how to bring in mail and packages sensors from various different delivery services to allow you to see your mail in the Media Browser as well as provide additional context to the delivery notifications.

A huge shout out to Ryan Hoffman who shared his experiences with the Ring mailbox sensor and how he was using the Camera Proxy Integration.

Configuring Integrations

In the sections below we will configure some integrations and email settings.

Installing Home Assistant Community Store (HACS)

If you haven't done so yet, please follow these instructions to install HACS. This will allow us to install and manage third party integrations, plugins, themes and more.

Installing the Media Source integration

Installing the Media Source integration will allow us to view what email is arriving today as well as any image snapshots we take when our mailbox is opened!

Installing Mail and Packages integration

Once HACS is installed, we need to install the Mail and Packages Integration. This integration uses IMAP to search your email account for specific emails by different delivery services like Amazon, USPS, FedEx, UPS, DHL and more.

There is a complete guide located on the projects site on how to install and configure the integration.

I really like receiving the daily USPS Informed Delivery email notifications with the images of mail being delivered so I can see if I even need to get the mail for today or I can wait until I get a package I care about. It's also helped me inquire when I didn't get a piece of mail it said was coming. Having this all in Home Assistant as an animated gif with sensors has taken it to a whole new level.

I recommend configuring the sensor list to track only what you are interested. This will ensure the sensors update quickly only searching your email for the delivery services you are most likely to use. This will create a bunch of sensors for each delivery service (e.g., sensor.mail_usps_mail, sensor.mail_amazon_packages).

Important: I search the all mail folder and archive all delivery email notifications. If you delete the notification emails, the sensors will not detect the arriving / delivered mail properly.

Finally, make sure you specify the image path to use the new media integration! You can do this by setting the Image Path to /media/mail. This will create a gif attachment /media/mail/mail_today.gif of all the mail being delivered to you that you can can view via the media browser or in a notification!

Here is what my configuration looked like:

Camera Integration

I recommend having a camera pointing where packages are normally dropped off (e.g., front porch) and mailbox if possible. This allows you to take a snapshot when a package is delivered and use these in notifications.

A smart doorbell with a video feed has been working great for my setup. I use the Ubiquiti UniFi Protect G4 Doorbell as it stores all the data locally. But you can use the Nest Hello or Ring Video Doorbell. Just be careful about your privacy settings and be aware of the cloud dependency.

Depending on your camera you choose, you'll need to configure it in Home Assistant.

Camera Proxy Integration

The Camera Proxy integration is a little known integration but worth it's weight in gold. This integration lets us crop a camera into only the area we are interested in, thus removing a ton of noise from an image. You make a camera containing only the area you are interested. Below we turn the original camera content

into

In order to setup the image cropping your first need to know the offsets in your existing image that you wish to crop. I'd start by saving an image snapshot from your camera feed and loading it into an image editing software like GIMP that supports showing you the X,Y coordinates of where you clicked.

Next, click on the upper left of where I wanted to start the crop. GIMP showed me the X1,Y1 of 750,580 when in the lower left toolbar. I then set image_top to 580 and image_left to 750.

Next, click on the lower left side where I wanted to end the crop. I got the X2,Y2 of 1015,705 when in the lower left toolbar. I then did some basic subtraction and set max_image_width and max_stream_width to the value of X2-X1 and set max_image_height and max_stream_height to the value of Y2-Y1

Here is the yaml configuration that I used for my camera:

- platform: proxy
  entity_id: camera.front_door
  name: Mailbox
  mode: crop
  max_image_width: 265
  max_image_height: 125
  max_stream_width: 265
  max_stream_height: 125
  image_top: 580
  image_left: 750

You can now reference this Camera Proxy integration as a normal camera named camera.mailbox!

Mailbox Sensor

If you already have a mailbox sensor you can skip this step.

I've researched a lot of different solutions both DYI and paid products for getting notifications when my mailbox was opened. Here was some of the things I was looking for in a sensor:

Will it last the brutally cold winters and thousands of open and closes?
Will it work in a metal faraday cage (e.g., metal mailbox)?
What is the battery life?
What is the max range the sensor data will transmit
Can I get the sensor data locally?
Will the company be around in five years?

Ring Mailbox Sensor

I ended up going with Ring Mailbox Sensor. I was really hesitant at first given Ring cloud dependency and track record around camera privacy. I'll cover this more in detail in a review later.

However, It was the only solution that worked with my metal mailbox due to the antenna and checked off most of the criteria.

The cable was also braided which gives me confidence it will last over time. I also wrapped the cable with some extra tape and taped it to the bottom of the mailbox and to the door.

I followed the Ring installation documentation and setup the light bridge as well.

Ring Integration

This integration assumes you are already using MQTT. I'd recommend using the official Mosquitto add-on.

Lets install the Ring Devices via MQTT Add-on from Home Assistant Community Add-ons repository. This allows us to run Ring discovery locally and have our devices integrate automatically with Home Assistant by taking advantage of all the awesome community work that's been done. Behind the scenes the add-on uses auto discovery to expose configured devices to Home Assistant automatically!

We have to install this addon because the official Ring integration doesn't support the Ring light bridge that the mailbox sensor uses.

Navigate to http://homeassistant.local:8123/hassio/store.
Click on the option menu in the upper right and select Repositories.
Enter https://github.com/tsightler/ring-mqtt-ha-addon and then click the Add button.
Click the close button to close the Manage add-on repositories dialog.
Select the Ring Devices add-on or access via the url http://homeassistant.local:8123/hassio/addon/03cabcc9_ring_mqtt/info.
Click the Install button.
Click the Start button.
Finally, open the Web UI once the addon is started and login to your Ring Account.

The Ring devices should now be discovered and shown under the MQTT Integration. I had two devices created for me and a mailbox sensor binary_sensor.mailbox_motion.

Automations

The possibilities are endless for what you can do when the mailbox is opened.

I'd recommend checking out Home Assistant Blueprints for quickly importing and consuming automations.

Send picture notification when mailbox is opened

I'm using the following Home Assistant Blueprint to send a notification when the binary_sensor.mailbox_motion is triggered using the camera.mailbox for the camera source.

I customized the Title to Mailbox opened and used a template for the Message of Expecting {{ states("sensor.mail_usps_mail") }} items to be delivered. This uses one of the sensors that our Mail and Package integration added.

Store a media snapshot when mailbox is opened.

I'm using the following Home Assistant Blueprint to take a camera snapshots and save them to /media/mail for easy viewing.

I set the File Path to /media/mail/last_opened.jpg and the Archive File Path to /media/mail/archive/opened_{{ now().strftime("%Y%m%d-%H%M%S") }}.jpg

Automation ideas

The possibilities are endless for what notifications you could do when the mailbox is opened like:

Play a Text-to-Speech (TTS) notification for You Got Mail to any media_player.
Stream the camera.mailbox feed to a media_player.
Turn on the outside lights if the mailbox is opened at night.
Send notifications when mail is scheduled to be delivered.
Trigger object recognition to see who is at your mailbox.

Lovelace integration

You can also use a picture card to display the last snapshot of who opened the mailbox /media/mail/last_opened.jpg as well as one for all the mail out for delivery today with the /media/mail/mail_today.gif source. I've opened a feature request to make this much easier!

Conclusion

I hope this article helps you be better informed about mail deliveries and what packages to expect are coming. It's helped give me more confidence and saved myself countless empty trips to the mailbox.

Disturbing trends in smart home products and services

Blake Niemyjski — Thu, 03 Dec 2020 00:00:00 GMT

Over the last decade there has been some really great software products that have been shut down with little to no warning. This is getting so common that someone created a website called Killed by Google. After Google shutdown Reader in 2013, I spent months trying to find a replacement. During this research I determined I should try to find solutions to services I use everyday that wouldn't be shut down in some board room over profit margins.

About five years ago, I started researching smart home products and services as well as automation software to use in my next home. I started to notice the following trends:

Software products and services being shutdown with no notice
No software updates for promised features or security fixes
Software updates that remove out of the box functionality (features/bricking)
Free software that suddenly requires a paid subscription
Companies are selling or hording your data.

For the sake of relevancy let's just cover what's happened since the start of 2019.

2019

ecobee

When ecobee launched the ecobee Switch+ I became super interested in them due to the shear amount of sensors in a small device. They had also stated there would be API support as well as it would be integrated as a remote temperature sensor for an ecobee smart thermostat.

It took most of 2019, before 15 of my ecobee switches got the firmware update to be added to my thermostat but the delivered promise of api support never came.

Best Buy Insignia shut down

Best Buy quietly announced on September 6th that it was shutting down it's Insignia brand of smart home devices on November 6th. App access would no longer be available rendering most of the products unusable.

The announcement has already been purged from the Best Buy Insignia website.

Lowe's Iris shut down

Lowe's Iris platform first launched in 2012, offered free and paid subscription options. An announcement on January 31st that it was shutting down on March 31st, rendering devices dead.

I'm happy they offered a program to trade in any purchased devices for a prepaid credit card so users could migrate to a different solution.

The announcement has already been purged from the Lowe's Iris website.

Chamberlain MyQ Breaking API Changes

When it comes to automating garage door openers there are few out of the box solutions that work and are as popular as the Chamberlain MyQ Smart Garage Door Opener.

An open source project created a python library that many other open source projects use to integrate with MyQ using the MyQ Api. Unfortunately the MyQ team has broken this api multiple times in 2019 causing automations to break.

I hesitate a little bit every time I see an MyQ app update on my phone. as I'm always wondering if this will trigger my opener firmware to be updated and break my home automations.

It's been a cat and mouse game between smart home developers and the MyQ team.

Stringify shut down

Stringify was launched in 2014 and bought by Comcast in 2017. Fast forward 18 months later to April 8th, Stringify announces via email it will be shutting down new sign ups immediately and all data will be deleted on July 8th.

Sonos bricking devices

In December it was uncovered that Sonos was bricking devices upon agreeing to recycle your device or by upgrading the speaker.

Recycle Mode is a state your device enters 21 days after recycling confirmation in the Sonos app. In Recycle Mode, all data is erased and the device is permanently deactivated so you can safely and securely dispose of it. Once a device is in Recycle Mode, it cannot be reactivated.

This decision was overturned in March 2020.

Works with Nest shut down

Google announced on May 7th that it would be shutting down Works with Nest on August 31st. It was only after massive backlash that they announced that they would continue to support integrations until they were available in the new API's backed by a single Google Account.

Current users have to be super careful not to migrate their existing Nest accounts to the new Google account for fear of losing integrations. I believe this migration and integration support for various hubs will be completed in 2021.

Wink product availability

For over five months users in the Wink ecosystem couldn't purchase any new Wink devices due to product availability issues. This lasted from November 2018 through March 2019.

2020

August Connect firmware

In early 2020, I received a firmware update that caused all five of my August Smart Lock Pro Connect Hubs from connecting to my Wi-Fi. Slowly but surely they've came online but every now and then I'll have one drop offline. Thankfully I have the Z-Wave version and access the locks that way.

Dark Sky shut down

On March 31st Apple acquired Dark Sky and announced it would be shutting down it's API at the end of 2021. A lot of smart home users used the API to power weather automations.

Apple did the right thing by giving users plenty of time to move off of the API.

ecobee

As stated in the 2019 section, when remote thermostat sensors support was rolled out to the ecobee Switch+ via an firmware update. A few months later it was removed with a second firmware update. Now only some of my switches are paired as a remote sensor to my ecobee smart thermostats.

To make matters worse there is an unresolved server side bug that keeps renaming any switch that has a custom defined name back to the serial number of the device. This should never happen to a consumer device paired with cloud outages leaves a lot to be desired from ecobee when it comes to software. I recommend using ecobee's HomeKit support to get around these limitations where possible.

Home Assistant breaking changes and OS changes

In May the Home Assistant team announced that they were dropping support for generic linux installs without consulting the community. This was a major change that left many users frustrated. It makes sense that users should be using the managed Home Assistant operating system, but a large portion of users installed it on Ubuntu and other linux operating systems of which may have been used for other purposes.

As the platform matures there will be less churn and breaking changes.

As we near the 1.0 release of Home Assistant, we've seen more integrations be removed as well as changes to the API surface. Although these are good changes they still require work by end users. This is to be expected of pre version 1.0 software.

Although breaking changes do happen from time to time. You control the updates on your own time, helping to limit breaking changes.

Chamberlain MyQ breaking API changes

For the second year running the Chamberlain MyQ Smart Garage Door Opener has had issues due to MyQ trying lock out users from using the MyQ API. It has been a cat and mouse game between smart home developers and the MyQ team.

Tuya firmware locking

I love flashing retail devices from the stock firmware to ESPHome. I do this mainly for privacy and to ensure it can be brought into Home Assistant easily. It appears there is a new firmware on some Tuya devices that is preventing flashing causing vendor lock in to unknown third party services. I can't blame companies for wanting to lock down the firmware, it just leaves little options for smart home integrations when the company leaves the space.

Check out this workaround for how to flash devices that are locked down.

Logitech Harmony Express remote shut down

On July 9th Logitech announced it was discontinuing the 15 month old smart remote control that cost $250. It would also be shutting down the Alexa service on September 30th, rendering the device useless to control by it's main selling feature.

Logitech is offering a free upgrade if you bought this product!

Wink Subscription

On May 20th Wink announced that it would start charging all users $4.99 a month. Failure to subscribe would result in severely limited functionality, pretty much forcing users to upgrade in order to add or edit devices, add automations, or use voice control.

IFTTT Subscription

On September 10th IFTTT announced a pro plan that costs $9.99 a month. The existing free plan would be severely limited to 3 applets.

Smart Things Classic shut down

On June 24th SmartThings announced that SmartThings classic will be shutting down on December 1st . All users who haven't migrated to the new SmartThings API will have major disruptions (e.g., your integrations will be deleted).

Wemo cloud only

On May 26th Wemo announced Wemo Accounts, a new way to manage all of your devices with an online account.

The new account system shifts all of the device management away from your local network and onto cloud services, which should reduce problems with the Wemo apps and integrations with third-party services. However, account creation is currently broken, which is very on-brand for Wemo.

OSRAM shut down

On March 9th, OSRAM announced that it would be shutting down it's smart lighting systems servers on August 31st, 2021. They provided a faq and recommended resetting all your devices and pair them to a Zigbee bridge.

Hue Bridge v1 shut down

On March 6th Hue announced that it would be shutting down internet connectivity for the first generation bridge on April 30th. You can still use the bridge locally but they recommended upgrading to the newer generation bridge.

Automatic shut down

On May 1st Automatic Labs announced that it would be shutting down it's vehicle-tracking service on May 28th

Android Things shut down

On December 18th Google announced that it would be shut down the sign up of Android Things new projects on January 5th, 2021. On January 5th, 2022 it will be shutting down the service completely.

Conclusion

It's worth looking at all the services and products that were shut down and disappeared like they never even existed. Some of these products and service had amassed hundreds of thousands of users. Is there an emerging pattern? Would you ever trust this company again?

I recommend buying devices that can be controlled locally though ESPHome, mqtt, Thread, Zigbee or Z-Wave. Stay away from products that use a vendor specific protocol especially if the company has a history of shutting down services or products. I'd also avoid large investments in new products by any company unless they have a positive track record or can be controlled via a standard.

If an online service doesn't charge you a monthly fee, you are the product. Products that have a free cloud dependency have to find a way to turn a profit or they will shut down. Someone has to pay for the cloud services.

I do feel comfortable sticking with HomeKit supported devices as I can control them locally and Apple has a good track record of privacy, supporting devices, and part of HomeKit was open sourced too.

Securing smart home devices with ESPHome and Home Assistant

Blake Niemyjski — Sat, 17 Oct 2020 00:00:00 GMT

When I first came across the ESPHome project a year or so ago. I thought it was interesting and something that I would use if I was building a smart device from scratch. I thought of all the things I could do, but I didn't really like the idea of all the maintenance I could foresee.

ESPHome is a system to control your ESP8266/ESP32 by simple yet powerful configuration files and control them remotely through Home Automation systems.

It took some persuasion by Ryan Hoffman, who has been building a bunch of custom made ESPHome projects to show me what is possible with ESPHome. I'm starting to kick myself in the butt (better now than never) that I didn't take time to check this project out sooner. I've since purchased multiple ESPHome devices such as the Gosund Mini WiFi Outlet and HiLetgo ESP-WROOM-32 ESP32. The next device that I think will work great for holiday lights is the Gosund Smart Power Strip.

I think one really huge benefit that mostly goes unnoticed for for a larger part of the smart home community is all the retail devices running ESPHome. This means that you can buy devices you might not trust on your networks due to them being from a company you've never heard from and completely flashing the firmware to something you've created. Yes, there still could be software embedded onto different chips but it's unlikely.

In the sections below I will cover installing ESPHome to work with Home Assistant, flashing a device, and adding the device to Home Assistant.

ESPHome can use mqtt so you can use it without Home Assistant too!

Benefits

Here are some of the benefits I've noticed since using some of my first devices.

Open Source (Audited)
Easy to manage with Over-the-air (OTA) updates
Flash any ESPHome device (including retail devices)
Cheap
Rock Solid
Rich local automation support right inside the firmware that can use Home Assistant states

Installing the ESPHome add-on

Lets install the ESPHome add-on from Home Assistant Community Add-ons repository. This allows us to run ESPHome locally and have our devices integrate automatically with Home Assistant by taking advantage of all the awesome community work that's been done. Behind the scenes the add-on uses auto discovery to expose configured devices to Home Assistant automatically!

There is an official getting started guide, but I found it slightly confusing and the steps below straight forward!

Navigate to http://homeassistant.local:8123/hassio/store.
Click on the option menu in the upper right and select Repositories.
Enter https://github.com/hassio-addons/repository and then click the Add button.
Click the close button to close the Manage add-on repositories dialog.
Select the ESPHome add-on or access via the url http://homeassistant.local:8123/hassio/addon/a0d7b954_esphome/info.
Click the Install button.
Toggle the Show in sidebar option for ease of use (Optional)
Finally, click the Start button.

ESPHome should now be running as an add-on and you can access the ESPHome web ui on via the sidebar or directly using http://homeassistant.local:8123/hassio/ingress/a0d7b954_esphome.

ESPHome devices

A huge misconception I had with ESPHome was that I had to build all my devices. It wasn't until recently that it was pointed out to me that there are a ton of cheap consumer devices from brands you've never heard of, and more than likely never have trusted to ever connect to your IoT/NoT network. This is super awesome game changer! You can build your own device firmware, flash the manufactures device knowing exactly what the device is doing, and then reflash it Over-the-air (OTA) via ESPHome with a few clicks! This really saves on the maintenance as well as having more trust for the security of the device.

My first ESPHome compatible device was the Gosund Mini WiFi Outlet

I'd recommend checking out one of the following links to find retail devices that work with ESPHome

Amazon ESPHome
Amazon 2.4GHz WiFi Only (2.4GHz WiFi Only is a really good indication a device is running ESPHome, but double check)
ESPHome Configs
TASMOTA devices (An ESPHome alternative)

Once you have a device we need to flash it, in order to flash it we need to configure some software and hardware.

Please note you may BRICK your device by flashing it or break the terms of any device/licenses.

Hardware Requirements

I used a spare Raspberry Pi 3 B+ that I had laying around, otherwise any Pi that supports Wifi will work like the Raspberry Pi 4 Model B.

Please note that you can use any free hardware granted you have an unused wifi adaptor. You just need to meet and it meets the requirements of tuya-convert.

Installing tuya-convert on a Pi

In this section I'll be configuring the Raspberry Pi to run the flashing software tuya-convert. Please follow the steps below to get tuya-convert up and running.

Download Raspberry Pi OS
Install the Raspberry Pi Imager
Insert a 2GB or bigger SD Card into your computer.
Launch the Raspberry Pi Imager and image the SD Card with the downloaded Raspberry Pi OS.
Remount the imaged SD Card.
Navigate to the root of the SD Card and create a file called ssh (e.g., touch ssh). This will allow you to SSH into the Pi in a headless manner.
Eject the SD Card and insert into your Pi.
Ensure you have ethernet plugged into your Pi
Power on the Pi and wait.
SSH into the Pi ssh pi@10.0.0.100 (you might need to figure out the ip using your router).
Enter in the default password of raspberry.
Install git using sudo apt-get install git and enter yes when prompted.
Install tuya-convert by cloning it with git clone https://github.com/ct-Open-Source/tuya-convert.git.
Navigate to the tuya-convert directory using cd tuya-convert.
Install the dependencies by running ./install_prereq.sh.

Now that tuya-convert is installed, we need to switch gears and build our custom firmware for our device.

Building the ESPHome Firmware

Now it's time to jump back into the ESPHome addon we configured earlier. You can access the ESPHome web ui from the sidebar or directly using http://homeassistant.local:8123/hassio/ingress/a0d7b954_esphome.

I'll be building the firmware for the Gosund Mini WiFi Outlet, if you are using a different ESPHome device, you will need to tweak the following steps slightly.

Create new ESPHome device

Click the Add button in the lower right hand side to bring up a Create New Node modal.
Enter in the new Node Name and click Next. I entered gosund_wp3_02 as this is my second device.
Select the device type and and click Next. I selected Generic ESP8266.
Enter in your WiFi network information or some garbage text (we will configure them via secrets in a little bit) and click Next

It's worth pointing out that you may see a red line at the top of your device. This means that your device is offline and hasn't communicated with ESPHome. If it is green, that means the device is online.

Also, if you look closely at this point to your Home Assistant installation, you will see that an esphome folder was created with a yaml file and folder that matches your Node Name! This allows you to make edits in your editor of choice and also commit these to source control!

Edit ESPHome Secrets

We will configure some secrets that will be merged into the firmware configuration. This works much like the lovelace secrets work! You can do this by clicking on the options menu in the upper right and selecting Secrets Editor.

It's also worth pointing out that you can have ESPHome use your Home Assistant secrets.yaml file!

The firmware I will be using uses the following secrets. This may be different for the firmware you are using.

wifi_ssid: "my_secret_network_name"
wifi_password: "my_secret_network_password"
wifi_fallback_password: "my_secret_network_password"
gosund_ota_password: ""

After you've configured the secrets hit Save and Close buttons respectively.

Edit ESPHome Device Firmware

Now that the secrets are configured lets click on the Edit button of the new device we created inside of ESPHome. I'm going to copy the firmware yaml from github into this editor and click Save and Close buttons respectively.

This firmware supports substitutions, which allows you to make variable replacements to the firmware easily. I use substitutions to change the device_name or friendly_name.

Next, click on the ESPHome Device options in the upper right of we just edited and select Compile. This will open a modal and start compiling the firmware. Once we the firmware is compiled there will be an option to Download Binary. Click this to save the firmware to your machine.

Uploading the firmware to the Pi

The next step could be the most trickiest step of them all (if you are running Windows), which is getting the firmware onto the pi. We need to copy the file from the downloads folder to the /home/pi/tuya-convert/files folder.

This can be done by navigating to the downloads folder in your terminal and entering: scp gosund_wp3_02.bin pi@10.0.0.100:/home/pi/tuya-convert/files and enter in the default password of raspberry when asked.

Flashing...

Don't forget to plug in our device and hit the pairing button to enter setup mode!

After the device is in setup mode, switch back over to the Pi that we were connected to earlier via SSH and run the ./start_flash.sh script

pi@raspberrypi:/home/pi/tuya-convert# ./start_flash.sh

You may have to enter y, to terminate some running service (e.g., dnsmasq, mosquitto) when prompted. After a while you will be presented with several options

Available options:
  0) return to stock
  1) flash gosund_wp3_02.bin
  q) quit; do nothing

Double check yourself at this step as you don't want to flash the wrong firmware!

You will want to flash it with the binary you uploaded in the previous step! Enter in the number of your firmware and follow any remaining prompts. After the device reboots it will now show up in ESPHome highlighted in green and you'll be asked to configure it in Home Assistant!

locked firmware

It appears there is a new firmware on some Tuya devices that is preventing flashing causing vendor lock in to unknown third party services. There are two workarounds, one requires soldering and the other solution requires a 3D Printer

</iframe>

Adding the flashed device to home assistant

The device will be automatically discovered by the ESPHome add-on and you'll be asked to configure the device.

Once you configure the device you'll have several device entities as shown below.

It's worth noting that you can use the physical button and the relay independently. This is possible because you have full control over how your devices operate when building your firmware!

Updating Device Firmware

If you need to update the device firmware later, it is a simple process. All you need to do is navigate to the device in the ESPHome Dashboard and click Edit. After you've made your changes, click the Upload button.

This will compile and upload the firmware to the device Over-the-air (OTA) and then start tailing the device logs. Just click the Stop button after the firmware is updated to close the modal.

Final Thoughts

I've really enjoyed using ESPHome so far. The overall ease of use and seamless integration into Home Assistant feels really polished. I've also noticed that all of my ESPHome devices are really stable and rock solid! I'll be buying more ESPHome devices for my projects.

Building and maintaining a smart home presentation

Blake Niemyjski — Tue, 25 Aug 2020 00:00:00 GMT

Here is a recent recording of a new talk I recently gave at North Dallas Developers on my experiences building and maintaining a smart home.

</iframe>

Here is a high level overview of what is covered in the presentation:

First Steps
- What are your Goals?
- Existing products and ecosystems
Research
- Future technologies
- Are you the product?
- How-to choose a contractor
- Costs
Accessibility
Breaking Changes
Companies will break you or sell your data
Smart Home Hubs
Automations

Feedback

Any feedback is greatly appreciated on how I can improve this talk. I had been wanting to give one on smart homes for a while so decided to create one over the weekend and present on it. Do you like the format? Was it helpful?

Speaking at North Dallas Developers

Blake Niemyjski — Tue, 04 Aug 2020 00:00:00 GMT

I'll be giving a talk at North Dallas Developers on Building and maintaining a smart home on August 5, 2020 at 6:30 PM Central Standard Time.

Visit the link above to RSVP to the virtual talk.

Overview

I'll be covering my experiences building and maintaining a smart home. Starting from nothing to having a fully automated house can be overwhelming. We’ll cover many topics like choosing future proof products (usability, security, etc..), dealing with contractors, keeping a happy spouse with ever changing automations and much more.

How-to: Using NFC Tags to trigger device or home automations

Blake Niemyjski — Sun, 19 Jul 2020 00:00:00 GMT

This post will cover how I use NFC tags with Home Assistant to trigger automations. I've laid out all the information below on how you can use NFC Tags to automate other apps and devices as well!

I really like using Near Field Communication (NFC) for home or device based automations because it is really cheap and the possibilities are endless as you can place them in any environmental condition and on almost any physical object.

Here are some of the ways you could use NFC Tags to trigger home automations:

Place a tag next to your bed to activate good morning/night routines.
Place a tag next to a door to automatically unlock a door (presence detection is hard).
Place a tag on the lawn mower and have an automation to open the Garage Door.
Place a tag under a bed to trigger an emergency action like trigger alarm.
Place a tag outside next to patio furniture to turn on lights or music.
Place a tag next to a lamp with a smart bulb to turn it on or off.
Place a tag next to the cat litter to trigger a future reminder to change it.

Here are some of the ways you could use NFC Tags to trigger device automations:

Sharing Guest Wifi information!
Place under a bed to trigger an emergency action like text emergency services.
Place a tag by your relaxation place and automatically play relaxing music.
Put one on your bag and tap it to trigger airplane mode or silence your phone.

A huge shout out to Ryan Hoffman who inspired me to look into using NFC after sharing his experiences with it. I'm not sure I would have stumbled upon NFC Tags otherwise.

What Tag Should I Get?

There are many different types of NFC tags that have different form factors, features (write locking, password protection, number of writes, etc).

I really like these heavy duty Type 2 NFC Tags as they are waterproof, safe for outdoor use and supports all of the NFC features you'd expect (locking, multi-write, password protection). Additionally, they come with 3M adhesive backing so you can literally stick them anywhere!

I could have done more research and found a tag that blended in better with my door, please keep this in mind when picking out where you want to place these.

You can get NFC Tags in sticker form as well. Just be sure it supports your all of your requirements. Here are some factors you might want to consider:

Location: Some tags are not waterproof and some don't work on metal surfaces.
Device support: Will this work with the devices I want to use it with (e.g., iPhone).
Updatable: Do you need to update this tag once installed (eg., Can I lock it, or write to it multiple times)?
Form Factor: Does this need to be a sticker?
Color: Does this blend in with the surrounding?

NFC Tag Tips

It's important to note that you should associate a name or identifier to a tag or to the event name you trigger. I'd strongly recommend creating tags with sequentially or with random characters like nfc_tag_1 and not with a specific name like office_lights_tag. The reason behind this is this because you can:

Potentially leak information publicly as it can be read by anyone with an NFC device.
Prevents you from reusing tags at a later date.

Finally, don't store anything you don't publicly want exposed (e.g., api keys, IP addresses, ...). Tags can be copied / cloned with enough work. I've yet to figure out how to encrypt the payload of a tag but I've seen encrypted tags mentioned around in my various searches. Even still I wouldn't put anything sensitive on a tag!

Triggering events when NFC is detected.

I'm pretty new to NFC but in my adventures I've only come across two ways to trigger Home Assistant events when an NFC Tag is detected. You can do it via the Shortcuts App or via deep/universal linking.

This article primarily focuses on iOS support as I don't have any android devices to try on. The following links may need to be slightly tweaked. Please leave a comment below on what worked for you and I'll update this article.

Please note that there are deep links for a wide range of applications that you could use NFC tags on.

Skip to the Conclusion if you are looking for my recommendations on what solution to go with.

Deep/Universal links

For simplicity sake a deep/universal link will open an application with some arguments contained in the link of which can trigger an action. You can read more about the differences here.

You can do a wide range of actions with links like Call Services, Fire Events or Send a Location.

The advantage of using Deep/Universal links is it allows any authorized user to execute a Home Assistant action with no configuration defined on device.

Deep link

The following Deep link homeassistant://fire_event/nfc_tag_1 will launch the Home Assistant application if it's installed, fire an event called nfc_tag_1 and show an alert dialog saying the nfc_tag_1 event has been called.

This is less than ideal as you don't really want a notification interrupting you. The application opening is jarring enough.

Deep link without notification

Luckily we can take advantage of a feature called x-callback-url. This allows us to trigger application actions without a notification. We simply need to modify the url to homeassistant://x-callback-url/fire_event?eventName=nfc_tag_1

Universal link

Deep links work great as long as the application you are linking to is installed. If it isn't, than the link will do nothing. This is one of the problems Universal links solve by redirecting you to the App Store if the application isn't currently installed.

NOTE: Universal links like https://www.home-assistant.io/ios/nfc/ are registered when you install the application and works offline!

Here is the Universal link you'd need to register https://www.home-assistant.io/ios/nfc/?url=homeassistant://x-callback-url/fire_event?eventName=nfc_tag_1 to fire the action without a notification.

Writing links to NFC Tag

I used the free application called NFC Tools which is also available on the Google Play Store.

Open the application and follow the steps below:

Select the Write button.
Select the Add a record button.
Select the URL / URI option.
Enter in your Deep/Universal link and select OK.
Select the Write / xx Bytes button to write the link to the NFC Tag.

When your phone gets close to the NFC tag it will launch a banner prompting you to click on it to activate the event.

This isn't ideal but it's in place for security reasons so your phone doesn't automatically execute tasks.

Shortcuts

Shortcuts is an iOS application made by apple that allows you do device based tasks or automations much like Tasker on Android.

Shortcuts define tasks and automations in the app that can get triggered several ways. Shortcut tasks can be shared across devices easily. However Shortcuts Automations cannot be shared and is currently the only way to trigger an action when an NFC tag is detected without any prompts.

The nice thing about Shortcuts is you can have it automate anything (e.g., texting, application actions, video calling and more).

Shortcut

There are two kind of Shortcuts you can create. The first is referred to a Shortcut and can be shared or triggered from automations or by NFC Tags. The second is called Shortcut Automations and those get triggered automatically based on some kind of action or state like coming into proximity to a NFC Tag. Shortcut Automations cannot currently be shared as of iOS 13.

Creating a new Shortcut

We first need to create a new Shortcut in order for a NFC tag to trigger it.

Open the Shortcuts application and follow the steps below:

Select the Create Shortcut button.
Select the Add Action button.
Select the Apps button and select Home Assistant.
Select the Fire Event option.
Enter in the event name you wish to trigger (e.g., nfc_tag_1).
Select Show More and enter in {} for the Event Data.
Select the Next button.
Enter in the event name as the Shortcut name (e.g., nfc_tag_1).
Select the Done button to save the Shortcut.

You can now select on the Shortcut button you just added to fire an event! We can now write to an NFC Tag to tell it to run the Shortcut we just added when the tag is read by our device.

Writing Shortcut to NFC Tag

I used the free application called NFC Tools to write the Shortcut to the NFC Tag.

Open the application and follow the steps below:

Select the Write button.
Select the Add a record button.
Select the Shortcut option.
Enter in the name of the Shortcut and select OK.
Select the Write / xx Bytes button to write the link to the NFC Tag.

When your phone gets close to the NFC tag it will launch a banner prompting you to click on it to activate the event.

This isn't ideal but it's in place for security reasons so your phone doesn't automatically execute tasks.

Shortcuts Automations

As stated previously, Automations are setup in the Shortcut apps and cannot be shared across devices. Which means this will become a lot of work to maintain if you have more than one user who will be using NFC Tag for automations. The plus side is you don't need to write anything to the NFC Tag to use these automations and they run without any user interaction. I think this is the biggest win of this approach.

Open the Shortcuts application and follow the steps below:

Select the Automation tab.
Select the Add icon button.
Select the Create Personal Automation button.
Select the NFC option under the Settings Category.
Select the Scan button and scan your NFC Tag.
Enter in the event name as the Shortcut name (e.g., nfc_tag_1).
Select the Next button.
Select the Apps button and select Home Assistant.
Select the Fire Event option.
Enter in the event name you wish to trigger (e.g., nfc_tag_1).
Select Show More and enter in {} for the Event Data.
Select the Next button.
Turn off Ask Before Running, unless you wish to be prompted.
Select the Done button to save the Automation.

When your phone gets close to the NFC tag it will automatically fire the event.

Home Assistant Automation

Here is a quick little Home Assistant Automation you can use that will get triggered when your NFC Tag is read and the event is fired (e.g., nfc_tag_1).

- alias: Toggle light when event is nfc_tag_1 event is triggered
  trigger:
  - event_data: {}
    event_type: nfc_tag_1
    platform: event
  action:
  - service: light.turn_on
    data:
      entity_id:
        - light.office

Note: this automation can be created very easily in Home Assistant Automations User Interface (UI).

Future Plans

Here are some of the things I want to try todo in the future that I think we may need to address in Home Assistant if it's not already available.

I'd like to find a way to ensure my tag can run from any device without leaking implementation details on how to execute. I think this comes down to potentially adding shared universal deep link that works on various platforms.

Taking this a step further, it would be really nice to have a named access token that any user can access to preform an action. I think of this as a named short link of which I define the action that it can perform server side (e.g., turn on this light or fire this action). These could even be used to anonymously share state as well! But this would expose your Home Assistance instance in some way.

Some further research is needed to see if I can stack NFC Tag actions to have multiple run at the same time or if only supported ones will be executed by a specific device (e.g., If I put Shortcuts first, will it be skipped on Android). This could be used to stack events that get fired or services called.

Finally, it would be nice to figure out how to encrypt my tags just so I can learn more!

Conclusion

UPDATE: Since writing this article Home Assistant has added the support for Tags. I'd recommend using Tags but it still requires you to click on a notification banner to to fire an event. You may want to use both Tags and Shortcut Automations together by first configuring Tags and then setting up the Shortcut Automation.

I'm going to be using the Universal Link route with all my tags as I define the configuration once on what the tag should do and anyone authenticated with the Home Assistant app will be able to fire events.

I do think that Shortcut Automations has the best user experience by automatically triggering actions when NFC Tags are detected. But they are not sharable/synced across devices or accounts makes this a non starter unless you will always own a single Apple Device or you just love maintaining automations long term. I do think I may use Shortcut Automations for some automations of which extra interaction is not ideal like activating good night routines without remembering to press on the banner.

NFC Tags can be used for Home Automation or even device specific tasks so go wild!

Extra reading

Here are some useful links that I came across in my NFC adventures:

How-to normalize home volume levels with Node-RED

Blake Niemyjski — Fri, 29 May 2020 00:00:00 GMT

This post is going to cover normalizing all media_player volume levels with Home Assistant and Node-RED. If you haven't already, I'd recommend reading my previous blog post on sending Text-to-Speech notifications which dives a bit deeper on the optional accessibility Subflow this Flow consumes.

This automation normalizes all volume levels in my home based on the time of day using the Amazon Alexa Media Player Integration to bring in all my Amazon Echo and ecobee Switch+ devices into Home Assistant.

This is one of my favorite automations in my house and I think you will enjoy it too! I wrote this automation for several reasons:

I wanted all speakers to be quiet at bed time. This prevents someone from getting blasted at 2:00 am in the morning.
In the morning after I've had my cup of coffee, I want to increase the volume to an acceptable level.
Automation allows for reuse via sub-flows. This means you can trigger it for other use cases like friends are over and music is playing, increase or decrease the volume.
One less thing I have to think about, or constantly touch to change. In todays pandemic, this is also one less thing you potentially have to clean!
You can use this same routine for increasing the volume when triggering an alarm.
All speakers are at the same volume level when playing music on every speaker.

This Automation Flow is comprised of two different Subflows.

Alexa Media Player Custom Component

If you haven't done so already, install the Alexa Media Player Custom Component if you are using Alexa, otherwise skip this section.

This addon is amazing as it normalizes all of your Alexa enabled devices such as the Amazon Echo or ecobee Switch+ into a media_player that can be used by Home Assistant.

Service to set volume level.

This Automation Flow controls the volume levels by calling the media_player.volume_set service. I'd recommend testing it first by opening the Home Assistant Developer tools and navigating to the Services tab.

Next, start playing some music on your media_player and call the media_player.volume_set service with a payload of:

entity_id: media_player.amazon_echo_plus
volume_level: 0.6

If you don't hear a volume change, try a different volume_level in case the the device is already at that level. If it's not then stop and check the logs. This will help diagnose why the device isn't working and save you lots of time before continuing.

Input Boolean to control automations

I highly recommend adding multiple Input Booleans that controls if a specific automation or all automations can run. It's always nice to be able to turn on or off a specific automation or all in the case of maintenance. Here are the three input_booleans that are used in this Automation Flow.

If you choose a different name, be sure to update the code below.

automation_enable:
  name: Enable Automations
  icon: mdi:home-automation

automation_normalize_volume:
  name: Automation - Automatically Normalize Volume throughout the day
  icon: mdi:volume-high

automation_notifications:
  name: Notify when an automation is triggered
  icon: mdi:home-automation

Set Volume Level Subflow

This Subflow will get the current devices volume level, compare the current and desired state. If changes are required, it will call the required service to change the devices volume level.

I then create a new Subflow with a status node, one input and one output. This Subflow optionally depends on the Send Automation Speech Notification Subflow.

graph TD
  InputNode(Input) -->  ACurrentStateNode(fas:fa-database Get Current Volume Level)
  ACurrentStateNode --> BFunctionNode(fas:fa-code Set Volume Level Payload)
  BFunctionNode --> CCallServiceNode(fas:fa-play Set Volume Level)
  BFunctionNode --> DCallSubflowNode(fas:fa-sliders-h Send Automation Speech Notification)
  CCallServiceNode --> OutputNode(Output 1)

  StatusNode(fas:fa-heartbeat Status: All) --> StatusOutputNode(Status)

style StatusNode fill:#ECF5FF
style StatusOutputNode fill:#FAFAFB

style InputNode fill:#FAFAFB
style ACurrentStateNode fill:#66ACFD
style BFunctionNode fill:#FBB68F
style CCallServiceNode fill:#66ACFD
style DCallSubflowNode fill:#FF75A1
style OutputNode fill:#FAFAFB
linkStyle default stroke-width:2px,fill:none,stroke:#CCD0D4

Let's break down the Subflow. The incoming message gets passed to the Get Current Volume Level Current State node to populate msg.data with the entity. This is then passed to the Set Volume Level Payload node which checks to see if any changes are required to meet the desired volume level and outputs Text-to-Speech (TTS) status messages. If changes are required, the Set Volume Level node will call the media_player.volume_set service.

Here is the JavaScript code contained in the Set Volume Level Payload Function node to smartly adjust the volume only if it is needed.

const entity = msg.payload && msg.payload.entity_id;
const attributes = msg.data && msg.data.attributes
if (!entity || !attributes) {
    node.status({ fill: "red", shape: "dot", text: "Invalid volume payload" });
    return [null, null];
}

const desired_volume_level = msg.payload.volume_level || 0;
if (desired_volume_level < 0.0 || desired_volume_level > 1.0) {
    let message = "Volume level must be between 0 and 1.";
    node.status({ fill: "red", shape: "dot", text: message });
    node.error(message);

    return [null, {
        payload: {
            entity_id: entity,
            message: message
        }
    }];
}

if (desired_volume_level === attributes.volume_level) {
    let message = "Volume level is already at the desired level.";
    node.status({ fill: "grey", shape: "dot", text: message });
    node.log(message);
    return [null, {
        payload: {
            entity_id: entity,
            message: message
        }
    }];
}

const volumePayload = {
    payload: {
        data: {
            entity_id: entity,
            volume_level: desired_volume_level
        }
    }
};

let message = "Setting Volume Level to " + (desired_volume_level * 100) + " percent.";
const speechPayload = {
    payload: {
        entity_id: entity,
        message: message
    }
};

node.status({ fill: "green", shape: "dot", text: message });
return [volumePayload, speechPayload];

As you an see it will output a friendly speech notification payload that will be sent to the Send Automation Speech Notification Subflow. If you are not using the alexa_media_player Home Assistant addon, you may want to update the you'll want to update Set Volume Level Payload Function node to take in a volume range you are expecting.

You can trigger this Subflow by passing a message object with the following payload. I recommend using a Inject Node to test this out.

{
  "entity_id": "media_player.amazon_echo_plus",
  "volume_level":0.4
}

Please note, for my Alexa speakers. I use a volume level between the range of [0.0..0.10). To mute, pass 0.0.

Set Volume Level on All Media Devices Subflow

This subflow will get all media_player' and then send them one by one to the Set Volume Level Subflow.

If you are not using Alexa, you may need to tweak the Get All Media Players node as it's returning all devices that have a specific feature attribute which says I support changing the volume level.

graph TD
  InputNode(Input) --> AFunctionNode(fas:fa-code Parse Volume Level)
  AFunctionNode --> BGetEntitiesNode(fas:fa-search Get All Media Players)
  BGetEntitiesNode --> CChangeNode(fas:fa-random Set Volume Level Payload)
  CChangeNode --> DSubflowNode(fas:fa-sliders-h Set Volume Level)
  DSubflowNode --> OutputNode(Output)

  StatusNode(fas:fa-heartbeat Status: All) --> StatusOutputNode(Status)

style StatusNode fill:#ECF5FF
style StatusOutputNode fill:#FAFAFB

style InputNode fill:#FAFAFB
style AFunctionNode fill:#FBB68F
style BGetEntitiesNode fill:#66ACFD
style CChangeNode fill:#FEA530
style DSubflowNode fill:#FF75A1
style OutputNode fill:#FAFAFB

linkStyle default stroke-width:2px,fill:none,stroke:#CCD0D4

You can trigger this Subflow by passing a message object with the following float (e.g. 0.1) payload. I recommend using a Inject Node to test this out.

Normalize Volume Levels Everywhere Flow

This flow brings the two previous sub flows together to normalize all volume levels based on the time of the day. In my home, I set the level to 30% from 9:00am to 9:00pm, otherwise I set to 10%.

To have the flow set the volume levels at different times of the day, I use the BigTimer Node with an On Time of 09:00 and a Off Time of 21:00. Then I set the ON Msg to .3 and a OFF Msg to .1. When the timer turns on and turns off, it will check to see if my automation flags are turned on. If they are, it will enumerate over all media devices and set the volume based on the timers output message (e.g., .1 or .3).

graph TD
  ABigTimerNode(fas:fa-clock Normalize Volume) --> BCurrentStateNode(fas:fa-database Automations Enabled?)
  BCurrentStateNode --> CCurrentStateNode(fas:fa-database Normalize Volume Levels?)
  CCurrentStateNode --> DRbeNode(fas:fa-scroll Only allow changed values)
  DRbeNode --> ECallSubflow(fas:fa-sliders-h Set Volume Level on All Media Devices Subflow)

  StatusNode(fas:fa-heartbeat Status: All) --> StatusOutputNode(Status)

style StatusNode fill:#ECF5FF
style StatusOutputNode fill:#FAFAFB

style ABigTimerNode fill:#3DB39F
style BCurrentStateNode fill:#66ACFD
style CCurrentStateNode fill:#66ACFD
style DRbeNode fill:#FEB95E
style ECallSubflow fill:#FF75A1

linkStyle default stroke-width:2px,fill:none,stroke:#CCD0D4

You can import this Flow and all Subflows shown above by importing the following JSON.

[
    {
        "id": "7a62a2d3.c0f4a4",
        "type": "Subflow",
        "name": "Send Automation Speech Notification",
        "info": "",
        "category": "",
        "in": [
            {
                "x": 60,
                "y": 100,
                "wires": [
                    {
                        "id": "87411254.a6ed18"
                    }
                ]
            }
        ],
        "out": [
            {
                "x": 1120,
                "y": 100,
                "wires": [
                    {
                        "id": "d44cc51a.be0668",
                        "port": 0
                    }
                ]
            }
        ],
        "status": {
            "x": 220,
            "y": 40,
            "wires": [
                {
                    "id": "aef53056.742438",
                    "port": 0
                }
            ]
        }
    },
    {
        "id": "89300b1.595bcf8",
        "type": "function",
        "z": "7a62a2d3.c0f4a4",
        "name": "Set Speech Payload",
        "func": "const entity = flow.get(\"$parent.speech_entity_id\") || (msg.payload && msg.payload.entity_id) || \"media_player.office_echo_plus\";\nconst message = (msg.payload && msg.payload.message) || \"Automation provided no message\";\nconst announcement = msg.payload && msg.payload.announcement;\n\nif (announcement) {\n    node.status({ fill: \"green\", shape: \"dot\", text: \"Announce message:\" + message });\n    return {\n        payload:{\n            data: {\n                message: message,\n                data: { \"type\": \"announce\", \"method\": \"all\" },\n                target: !!entity ? [entity] : []\n            }\n        }\n    };\n}\n\nnode.status({ fill: \"green\", shape: \"dot\", text: \"TTS message:\" + message });\nreturn {\n    payload:{\n        data: {\n            message: message,\n            data: { type: \"tts\" },\n            target: [entity]\n        }\n    }\n};",
        "outputs": 1,
        "noerr": 0,
        "x": 700,
        "y": 100,
        "wires": [
            [
                "d44cc51a.be0668"
            ]
        ]
    },
    {
        "id": "d44cc51a.be0668",
        "type": "api-call-service",
        "z": "7a62a2d3.c0f4a4",
        "name": "Send Speech Notification",
        "server": "61956bd4.93df44",
        "version": 1,
        "debugenabled": false,
        "service_domain": "notify",
        "service": "alexa_media",
        "entityId": "",
        "data": "",
        "dataType": "json",
        "mergecontext": "",
        "output_location": "payload",
        "output_location_type": "msg",
        "mustacheAltTags": false,
        "x": 950,
        "y": 100,
        "wires": [
            []
        ]
    },
    {
        "id": "6add9bc6.4c3624",
        "type": "api-current-state",
        "z": "7a62a2d3.c0f4a4",
        "name": "Speech Notifications?",
        "server": "61956bd4.93df44",
        "version": 1,
        "outputs": 2,
        "halt_if": "true",
        "halt_if_type": "bool",
        "halt_if_compare": "is",
        "override_topic": false,
        "entity_id": "input_boolean.automation_notifications",
        "state_type": "habool",
        "state_location": "",
        "override_payload": "none",
        "entity_location": "",
        "override_data": "none",
        "blockInputOverrides": false,
        "x": 440,
        "y": 140,
        "wires": [
            [
                "89300b1.595bcf8"
            ],
            []
        ],
        "outputLabels": [
            "",
            "enabled"
        ]
    },
    {
        "id": "aef53056.742438",
        "type": "status",
        "z": "7a62a2d3.c0f4a4",
        "name": "",
        "scope": null,
        "x": 100,
        "y": 40,
        "wires": [
            []
        ]
    },
    {
        "id": "87411254.a6ed18",
        "type": "function",
        "z": "7a62a2d3.c0f4a4",
        "name": "Check for overrides",
        "func": "const alwaysSpeak = msg.payload && msg.payload.announcement\nif (alwaysSpeak) {\n    return [msg, null];\n} else {\n    return [null, msg];\n}",
        "outputs": 2,
        "noerr": 0,
        "x": 210,
        "y": 100,
        "wires": [
            [
                "89300b1.595bcf8"
            ],
            [
                "6add9bc6.4c3624"
            ]
        ],
        "outputLabels": [
            "Bypass notification",
            "Check for notifications"
        ]
    },
    {
        "id": "974adda.d2ea92",
        "type": "Subflow",
        "name": "Set Volume Level",
        "info": "Set Volume Level",
        "category": "",
        "in": [
            {
                "x": 80,
                "y": 120,
                "wires": [
                    {
                        "id": "4c86fd47.0875e4"
                    }
                ]
            }
        ],
        "out": [
            {
                "x": 960,
                "y": 100,
                "wires": [
                    {
                        "id": "263d05a1.5b6672",
                        "port": 0
                    }
                ]
            }
        ],
        "status": {
            "x": 240,
            "y": 40,
            "wires": [
                {
                    "id": "7a662cd5.162a24",
                    "port": 0
                }
            ]
        }
    },
    {
        "id": "563c252e.ce31e4",
        "type": "function",
        "z": "974adda.d2ea92",
        "name": "Set Volume Level Payload",
        "func": "const entity = msg.payload && msg.payload.entity_id;\nconst attributes = msg.data && msg.data.attributes\nif (!entity || !attributes) {\n    node.status({ fill: \"red\", shape: \"dot\", text: \"Invalid volume payload\" });\n    return [null, null];\n}\n\nconst desired_volume_level = msg.payload.volume_level || 0;\nif (desired_volume_level < 0.0 || desired_volume_level > 1.0) {\n    let message = \"Volume level must be between 0 and 1.\"; \n    node.status({ fill: \"red\", shape: \"dot\", text: message });\n    node.error(message);\n    \n    return [null, { \n        payload: {\n            entity_id: entity,\n            message: message\n        }\n    }];\n}\n\nif (desired_volume_level === attributes.volume_level) {\n    let message = \"Volume level is already at the desired level.\"; \n    node.status({ fill: \"grey\", shape: \"dot\", text: message });\n    node.log(message);\n    return [null, { \n        payload: {\n            entity_id: entity,\n            message: message\n        }\n    }];\n}\n\nconst volumePayload = { \n    payload: {\n        data: {\n            entity_id: entity,\n            volume_level: desired_volume_level\n        } \n    }\n};\n\nlet message = \"Setting Volume Level to \" + (desired_volume_level * 100) + \" percent.\";\nconst speechPayload = { \n    payload: {\n        entity_id: entity,\n        message: message\n    }\n};\n\nnode.status({ fill: \"green\", shape: \"dot\", text: message });\nreturn [volumePayload, speechPayload];",
        "outputs": 2,
        "noerr": 0,
        "x": 510,
        "y": 120,
        "wires": [
            [
                "263d05a1.5b6672"
            ],
            [
                "b41308c0.f41e9"
            ]
        ],
        "inputLabels": [
            "Volume Percentage"
        ],
        "outputLabels": [
            "Volume Level Payload",
            "Speech Notification Payload"
        ]
    },
    {
        "id": "263d05a1.5b6672",
        "type": "api-call-service",
        "z": "974adda.d2ea92",
        "name": "Set Volume Level",
        "server": "61956bd4.93df44",
        "version": 1,
        "debugenabled": false,
        "service_domain": "media_player",
        "service": "volume_set",
        "entityId": "",
        "data": "",
        "dataType": "json",
        "mergecontext": "",
        "output_location": "payload",
        "output_location_type": "msg",
        "mustacheAltTags": false,
        "x": 770,
        "y": 100,
        "wires": [
            []
        ]
    },
    {
        "id": "b41308c0.f41e9",
        "type": "Subflow:7a62a2d3.c0f4a4",
        "z": "974adda.d2ea92",
        "name": "",
        "env": [],
        "x": 830,
        "y": 160,
        "wires": [
            []
        ]
    },
    {
        "id": "4c86fd47.0875e4",
        "type": "api-current-state",
        "z": "974adda.d2ea92",
        "name": "Get Current Volume Level",
        "server": "61956bd4.93df44",
        "version": 1,
        "outputs": 1,
        "halt_if": "",
        "halt_if_type": "str",
        "halt_if_compare": "is",
        "override_topic": true,
        "entity_id": "",
        "state_type": "str",
        "state_location": "",
        "override_payload": "none",
        "entity_location": "data",
        "override_data": "msg",
        "blockInputOverrides": false,
        "x": 250,
        "y": 120,
        "wires": [
            [
                "563c252e.ce31e4"
            ]
        ]
    },
    {
        "id": "7a662cd5.162a24",
        "type": "status",
        "z": "974adda.d2ea92",
        "name": "",
        "scope": null,
        "x": 120,
        "y": 40,
        "wires": [
            []
        ]
    },
    {
        "id": "76be143e.dbb0e4",
        "type": "Subflow",
        "name": "Set Volume Level on All Media Devices",
        "info": "",
        "category": "",
        "in": [
            {
                "x": 100,
                "y": 120,
                "wires": [
                    {
                        "id": "642c5f33.bd92a8"
                    }
                ]
            }
        ],
        "out": [
            {
                "x": 1120,
                "y": 120,
                "wires": [
                    {
                        "id": "15d5ddae.8f64c2",
                        "port": 0
                    }
                ]
            }
        ],
        "env": [],
        "color": "#DDAA99",
        "status": {
            "x": 1120,
            "y": 40,
            "wires": [
                {
                    "id": "ba88f8d5.b5d648",
                    "port": 0
                },
                {
                    "id": "15d5ddae.8f64c2",
                    "port": 0
                }
            ]
        }
    },
    {
        "id": "f467a711.24ca3",
        "type": "ha-get-entities",
        "z": "76be143e.dbb0e4",
        "server": "61956bd4.93df44",
        "name": "Get All Media Players",
        "rules": [
            {
                "property": "attributes.supported_features",
                "logic": "is",
                "value": "56253",
                "valueType": "num"
            },
            {
                "property": "attributes.available",
                "logic": "is",
                "value": "true",
                "valueType": "bool"
            }
        ],
        "output_type": "split",
        "output_empty_results": true,
        "output_location_type": "msg",
        "output_location": "payload",
        "output_results_count": 1,
        "x": 480,
        "y": 120,
        "wires": [
            [
                "1fe46c89.0e0543"
            ]
        ]
    },
    {
        "id": "1fe46c89.0e0543",
        "type": "change",
        "z": "76be143e.dbb0e4",
        "name": "Set Volume Level Payload",
        "rules": [
            {
                "t": "set",
                "p": "payload.volume_level",
                "pt": "msg",
                "to": "volume_level",
                "tot": "flow"
            }
        ],
        "action": "",
        "property": "",
        "from": "",
        "to": "",
        "reg": false,
        "x": 730,
        "y": 120,
        "wires": [
            [
                "15d5ddae.8f64c2"
            ]
        ]
    },
    {
        "id": "642c5f33.bd92a8",
        "type": "function",
        "z": "76be143e.dbb0e4",
        "name": "Parse Volume Level",
        "func": "const volumeLevel = parseFloat(msg.payload);\nnode.status({ fill: \"green\", shape: \"dot\", text: \"Volume Level: \" + volumeLevel });\nflow.set(\"volume_level\", volumeLevel);\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "x": 250,
        "y": 120,
        "wires": [
            [
                "f467a711.24ca3"
            ]
        ]
    },
    {
        "id": "15d5ddae.8f64c2",
        "type": "Subflow:974adda.d2ea92",
        "z": "76be143e.dbb0e4",
        "name": "",
        "env": [],
        "x": 970,
        "y": 120,
        "wires": [
            []
        ]
    },
    {
        "id": "ba88f8d5.b5d648",
        "type": "status",
        "z": "76be143e.dbb0e4",
        "name": "",
        "scope": null,
        "x": 140,
        "y": 40,
        "wires": [
            []
        ]
    },
    {
        "id": "3c07a459.ce903c",
        "type": "tab",
        "label": "Normalize Volume",
        "disabled": false,
        "info": "Normalize All Volume"
    },
    {
        "id": "e949c86f.c9ca9",
        "type": "api-current-state",
        "z": "3c07a459.ce903c",
        "name": "Automations Enabled?",
        "server": "61956bd4.93df44",
        "version": 1,
        "outputs": 2,
        "halt_if": "true",
        "halt_if_type": "bool",
        "halt_if_compare": "is",
        "override_topic": false,
        "entity_id": "input_boolean.automation_enable",
        "state_type": "habool",
        "state_location": "",
        "override_payload": "none",
        "entity_location": "",
        "override_data": "none",
        "blockInputOverrides": false,
        "x": 560,
        "y": 100,
        "wires": [
            [
                "ec5308f5.aefff"
            ],
            []
        ],
        "outputLabels": [
            "",
            "enabled"
        ]
    },
    {
        "id": "ec5308f5.aefff",
        "type": "api-current-state",
        "z": "3c07a459.ce903c",
        "name": "Normalize Volume Levels?",
        "server": "61956bd4.93df44",
        "version": 1,
        "outputs": 2,
        "halt_if": "true",
        "halt_if_type": "bool",
        "halt_if_compare": "is",
        "override_topic": false,
        "entity_id": "input_boolean.automation_normalize_volume",
        "state_type": "habool",
        "state_location": "",
        "override_payload": "none",
        "entity_location": "",
        "override_data": "none",
        "blockInputOverrides": false,
        "x": 810,
        "y": 100,
        "wires": [
            [
                "bde48ad8.814ac"
            ],
            []
        ],
        "outputLabels": [
            "",
            "enabled"
        ]
    },
    {
        "id": "dbf47096.3a7dc",
        "type": "inject",
        "z": "3c07a459.ce903c",
        "name": "",
        "topic": "",
        "payload": "on",
        "payloadType": "str",
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "x": 110,
        "y": 80,
        "wires": [
            [
                "4608374e.2fd4d8"
            ]
        ]
    },
    {
        "id": "6eace9df.a5aa68",
        "type": "inject",
        "z": "3c07a459.ce903c",
        "name": "",
        "topic": "",
        "payload": "off",
        "payloadType": "str",
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "x": 110,
        "y": 120,
        "wires": [
            [
                "4608374e.2fd4d8"
            ]
        ]
    },
    {
        "id": "4608374e.2fd4d8",
        "type": "bigtimer",
        "z": "3c07a459.ce903c",
        "outtopic": "",
        "outpayload1": ".3",
        "outpayload2": ".1",
        "name": "Normalize Volume",
        "comment": "Set volume to 30% from 9:00am to 9:00pm, otherwise set to 10%",
        "lat": "44.50",
        "lon": "-88.06",
        "starttime": "540",
        "endtime": "1260",
        "startoff": "0",
        "endoff": 0,
        "startoff2": "",
        "endoff2": "",
        "offs": 0,
        "outtext1": "",
        "outtext2": "",
        "timeout": "240",
        "sun": true,
        "mon": true,
        "tue": true,
        "wed": true,
        "thu": true,
        "fri": true,
        "sat": true,
        "jan": true,
        "feb": true,
        "mar": true,
        "apr": true,
        "may": true,
        "jun": true,
        "jul": true,
        "aug": true,
        "sep": true,
        "oct": true,
        "nov": true,
        "dec": true,
        "day1": 0,
        "month1": 0,
        "day2": 0,
        "month2": 0,
        "day3": 0,
        "month3": 0,
        "day4": 0,
        "month4": 0,
        "day5": 0,
        "month5": 0,
        "day6": 0,
        "month6": 0,
        "day7": "",
        "month7": "",
        "day8": "",
        "month8": "",
        "day9": "",
        "month9": "",
        "day10": "",
        "month10": "",
        "day11": "",
        "month11": "",
        "day12": "",
        "month12": "",
        "d1": 0,
        "w1": 0,
        "d2": 0,
        "w2": 0,
        "d3": 0,
        "w3": 0,
        "d4": 0,
        "w4": 0,
        "d5": 0,
        "w5": 0,
        "d6": 0,
        "w6": 0,
        "xday1": "0",
        "xmonth1": "0",
        "xday2": "0",
        "xmonth2": "0",
        "xday3": 0,
        "xmonth3": 0,
        "xday4": 0,
        "xmonth4": 0,
        "xday5": 0,
        "xmonth5": 0,
        "xday6": 0,
        "xmonth6": 0,
        "xd1": 0,
        "xw1": 0,
        "xd2": 0,
        "xw2": 0,
        "xd3": 0,
        "xw3": 0,
        "xd4": 0,
        "xw4": 0,
        "xd5": 0,
        "xw5": 0,
        "xd6": 0,
        "xw6": 0,
        "suspend": false,
        "random": false,
        "repeat": false,
        "atstart": false,
        "odd": false,
        "even": false,
        "x": 340,
        "y": 100,
        "wires": [
            [
                "e949c86f.c9ca9"
            ],
            [],
            []
        ]
    },
    {
        "id": "418865ec.975eec",
        "type": "inject",
        "z": "3c07a459.ce903c",
        "name": "",
        "topic": "",
        "payload": "auto",
        "payloadType": "str",
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "x": 110,
        "y": 160,
        "wires": [
            [
                "4608374e.2fd4d8"
            ]
        ]
    },
    {
        "id": "e314dd72.39afe",
        "type": "inject",
        "z": "3c07a459.ce903c",
        "name": "",
        "topic": "",
        "payload": "manual",
        "payloadType": "str",
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "x": 110,
        "y": 200,
        "wires": [
            [
                "4608374e.2fd4d8"
            ]
        ]
    },
    {
        "id": "bde48ad8.814ac",
        "type": "rbe",
        "z": "3c07a459.ce903c",
        "name": "Only allow changed values",
        "func": "rbe",
        "gap": "",
        "start": "",
        "inout": "out",
        "property": "payload",
        "x": 480,
        "y": 180,
        "wires": [
            [
                "4173c48c.381f84"
            ]
        ]
    },
    {
        "id": "4173c48c.381f84",
        "type": "Subflow:76be143e.dbb0e4",
        "z": "3c07a459.ce903c",
        "name": "",
        "env": [],
        "x": 790,
        "y": 180,
        "wires": [
            []
        ]
    },
    {
        "id": "d45759b3.b76fe8",
        "type": "comment",
        "z": "3c07a459.ce903c",
        "name": "Set volume to 30% from 9:00am to 9:00pm, otherwise set to 10%",
        "info": "",
        "x": 270,
        "y": 40,
        "wires": []
    },
    {
        "id": "61956bd4.93df44",
        "type": "server",
        "z": "",
        "name": "Home Assistant",
        "legacy": false,
        "addon": true,
        "rejectUnauthorizedCerts": true,
        "ha_boolean": "y|yes|true|on|home|open",
        "connectionDelay": true
    }
]

NodeRed VS Home Assistant Automations

I personally prefer writing all my automations in NodeRed as it helps me visually see the flow of the automation. I'd love to hear what you like and if you'd like to see automations written using the built in Home Assistant Automations.

How-to send Text-to-Speech notifications with Home Assistant and Node-RED

Blake Niemyjski — Thu, 30 Apr 2020 00:00:00 GMT

Have you ever wanted to send Text-to-Speech (TTS) notifications with Home Assistant and Node-RED? In the sections below I'm going to be using Amazon Alexa Media Player Integration to send TTS notifications. Please note you can easily switch out the TTS integration to the one of your choosing and still use this Subflow (with a few tweaks).

This is one of my favorite Subflows as it can be used easily in a lot of automations. I use it in my automations to announce:

Friendly state change notifications for accessibility (e.g., motion detected in room abc, lights turning on in the current room, the garage door was unlocked).
General announcements (e.g., welcome to our home, it's bedtime).
Weather related announcements and warnings.

If you are new to Node-RED, I'd recommend checking out this awesome YouTube Node-RED playlist to get up to speed!

Alexa Media Player Custom Component

If you haven't done so already, install the Alexa Media Player Custom Component if you are using Alexa, otherwise skip this section.

This addon is amazing as it normalizes all of your Alexa enabled devices such as the Amazon Echo or ecobee Switch+ into a media_player that can be used by Home Assistant. Once configured you can simply verify the device you want to send a message is working by calling the notify.alexa_media service with the payload:

message: "Text-to-Speech is easy"
data:
  type: "tts"
  target: ["media_player.amazon_echo_plus"]

If you don't hear anything after calling this service, stop and check the logs. This will help diagnose why you are not getting TTS notifications before continuing.

Input Boolean to control automations

I highly recommend adding an Input Boolean that controls if audible notifications occur. It could get pretty annoying if you are constantly hearing notifications for automations that are running unless it's desired. Here is my input_boolean that I use for this automation. If you choose a different name, be sure to update the code below.

automation_notifications:
  name: Notify when an automation is triggered
  icon: mdi:home-automation

Send Automation Speech Notification Subflow

Lets open Node-RED and create a new Subflow with a Status Node, Input Node and Output Node. I really like using the Status Node to capture all statuses as it allows for a better experience using Subflows by showing you a visual indication of the current status. Also, I like specifying an Output Node as it allows a node to be part of a series when the Subflow has completed. It's worth noting that this Output Node will only be called if the notification is sent.

graph TD
  InputNode(Input) --> AFunctionNode(fas:fa-code Check For Overrides)
  AFunctionNode --> BCurrentStateNode(fas:fa-database Speech Notifications?)
  AFunctionNode --> |Announcement Override| CFunctionNode(fas:fa-code Set Speech Payload)
  BCurrentStateNode --> CFunctionNode
  CFunctionNode --> DCallServiceNode(fas:fa-play Send Speech Notification)
  DCallServiceNode --> OutputNode(Output 1)

  StatusNode(fas:fa-heartbeat Status: All) --> StatusOutputNode(Status)

style StatusNode fill:#ECF5FF
style StatusOutputNode fill:#FAFAFB

style InputNode fill:#FAFAFB
style AFunctionNode fill:#FBB68F
style BCurrentStateNode fill:#66ACFD
style CFunctionNode fill:#FBB68F
style DCallServiceNode fill:#66ACFD
style OutputNode fill:#FAFAFB
linkStyle default stroke-width:2px,fill:none,stroke:#CCD0D4

Let's break down the Subflow. The incoming message gets passed to the Check For Overrides node. This node simply checks to see if the msg.payload.announcement is set to true. If true, then it will always send the TTS notification. Otherwise it will call the Speech Notifications? Current State Node to see if the input_boolean.automation_notifications is true. To recap, we will only call Set Speech Payload if it an announcement is true or our input_boolean.automation_notifications is true.

The Set Speech Payload node will return a new payload that the Send Speech Notification Service Call Node is expecting. If you are not using the alexa_media_player Home Assistant addon, you'll want to update both of these nodes for the correct payload and service call.

You can then trigger this Subflow by passing a message object with the following payload. I recommend using a Inject Node to test this out.

{
    "announcement": true,
    "entity_id": "media_player.amazon_echo_plus",
    "message": "test"
}

When triggering this Subflow it's a good idea to pick the closest speaker to where the automation is occurring.

Here is the exported Node-RED subflow.

[
    {
        "id": "7a62a2d3.c0f4a4",
        "type": "subflow",
        "name": "Send Automation Speech Notification",
        "info": "",
        "category": "",
        "in": [
            {
                "x": 60,
                "y": 100,
                "wires": [
                    {
                        "id": "87411254.a6ed18"
                    }
                ]
            }
        ],
        "out": [
            {
                "x": 1120,
                "y": 100,
                "wires": [
                    {
                        "id": "d44cc51a.be0668",
                        "port": 0
                    }
                ]
            }
        ],
        "status": {
            "x": 220,
            "y": 40,
            "wires": [
                {
                    "id": "aef53056.742438",
                    "port": 0
                }
            ]
        }
    },
    {
        "id": "89300b1.595bcf8",
        "type": "function",
        "z": "7a62a2d3.c0f4a4",
        "name": "Set Speech Payload",
        "func": "const entity = flow.get(\"$parent.speech_entity_id\") || (msg.payload && msg.payload.entity_id) || \"media_player.amazon_echo_plus\";\nconst message = (msg.payload && msg.payload.message) || \"Automation provided no message\";\n\nnode.status({ fill: \"green\", shape: \"dot\", text: \"TTS message:\" + message });\nreturn {\n    payload:{\n        data: {\n            message: message,\n            data: { type: \"tts\" },\n            target: [entity]\n        }\n    }\n};",
        "outputs": 1,
        "noerr": 0,
        "x": 700,
        "y": 100,
        "wires": [
            [
                "d44cc51a.be0668"
            ]
        ]
    },
    {
        "id": "d44cc51a.be0668",
        "type": "api-call-service",
        "z": "7a62a2d3.c0f4a4",
        "name": "Send Speech Notification",
        "server": "61956bd4.93df44",
        "version": 1,
        "debugenabled": false,
        "service_domain": "notify",
        "service": "alexa_media",
        "entityId": "",
        "data": "",
        "dataType": "json",
        "mergecontext": "",
        "output_location": "payload",
        "output_location_type": "msg",
        "mustacheAltTags": false,
        "x": 950,
        "y": 100,
        "wires": [
            []
        ]
    },
    {
        "id": "6add9bc6.4c3624",
        "type": "api-current-state",
        "z": "7a62a2d3.c0f4a4",
        "name": "Speech Notifications?",
        "server": "61956bd4.93df44",
        "version": 1,
        "outputs": 2,
        "halt_if": "true",
        "halt_if_type": "bool",
        "halt_if_compare": "is",
        "override_topic": false,
        "entity_id": "input_boolean.automation_notifications",
        "state_type": "habool",
        "state_location": "",
        "override_payload": "none",
        "entity_location": "",
        "override_data": "none",
        "blockInputOverrides": false,
        "x": 440,
        "y": 140,
        "wires": [
            [
                "89300b1.595bcf8"
            ],
            []
        ],
        "outputLabels": [
            "",
            "enabled"
        ]
    },
    {
        "id": "aef53056.742438",
        "type": "status",
        "z": "7a62a2d3.c0f4a4",
        "name": "",
        "scope": null,
        "x": 100,
        "y": 40,
        "wires": [
            []
        ]
    },
    {
        "id": "87411254.a6ed18",
        "type": "function",
        "z": "7a62a2d3.c0f4a4",
        "name": "Check for overrides",
        "func": "const alwaysSpeak = msg.payload && msg.payload.announcement\nif (alwaysSpeak) {\n    return [msg, null];\n} else {\n    return [null, msg];\n}",
        "outputs": 2,
        "noerr": 0,
        "x": 210,
        "y": 100,
        "wires": [
            [
                "89300b1.595bcf8"
            ],
            [
                "6add9bc6.4c3624"
            ]
        ],
        "outputLabels": [
            "Bypass notification",
            "Check for notifications"
        ]
    },
    {
        "id": "61956bd4.93df44",
        "type": "server",
        "z": "",
        "name": "Home Assistant",
        "legacy": false,
        "addon": true,
        "rejectUnauthorizedCerts": true,
        "ha_boolean": "y|yes|true|on|home|open",
        "connectionDelay": true
    }
]

Conclusion

I hope this article inspired you to create your own reusable Subflows that you can use in your various automations. I've found this Subflow to be super useful, let me know how it works out for you!

Let's talk about smart home accessibility

Blake Niemyjski — Thu, 26 Mar 2020 00:00:00 GMT

I define accessibility as a critical necessity to be inclusive to everyone. No matter how big or small the task, anyone should be able complete the task at hand regardless of the disability (e.g., vision, hearing, mobility, device access, language, etc).

You might be wondering what does this have to do with smart homes and why should I think or care about this? I think there are a few simple facts to consider:

You may already have a disability.
You can dramatically improve the quality of life, safety and happiness of those who have a disability when they are in your home.
You never know when yourself or a loved one may have a disability. You could one day wake up and not be able to walk, see or hear (I'm serious). You could have a stroke or get hit by a car dramatically changing your way of life only to realize how unaccessible the world really is.
We are aging every day and everyone will have some type of disability at some point in your life.
If all consumers think about this, it will be for the betterment of all while bringing down costs.

I hope this showcased a different perspective on life! Now, I'm not saying we should all get rid of our current smart devices or do major construction for ramps, handrails or low countertops. But it is something you should keep in mind when buying new products, buying a house or doing a remodel. Did you ever consider this could be a reason why a house with a first floor bedroom is worth more than a house with a second story bedroom?

Also we can use our existing smart products (e.g., smart bulbs) to make a home more accessible with a little bit of time. I'll cover this below in the following sections.

Progressive Enhancement

Think of a single task or physical item that does something really well like a light switch with on and off actions. A progressive enhancement would be to add additional functionality without compromising the previous task/item. For example, a switch or light should always be able to be turned on like any other device, but it may be able to provide additional functionality like a night light, motion detection, or play multimedia.

Automations are a progress enhancement and can improve accessibility.

Think about the ways you can use these added enhancements in accessible ways but don't compromise on the main functionality of the device!

Switches

TLDR: Find a switch that will work great as just a switch, supports multiple protocols if possible (e.g., API and HomeKit, Z-Wave) and has progressive enhancements (motion, light) that don't take away from using the switch. Use These sensors to create automations that can improve accessibility.

The single responsibility of a switch is to turn on and off a device. This should continue to work unimpeded. A guest shouldn't know they are interacting with a smart home, it should work like any other home to be accessible. Switches with motion detection and smart home control can greatly increase the quality and reduce the physical need to turn lights or devices on or off.

I've seen multiple products like wall switch guards and 3D printed guards which only add to the problems.

These guards were put into place because turning off the switch would cause products like the Hue Smart Bulbs from working as smart bulbs need constant power. The real solution to this problem would be to use the official dimmer switch or even better, replace the switch with a smart switch that is designed to act like a normal switch but can also supply constant power to the smart bulb. One switch that fits this bill is the Inovelli Smart Switch. With this switch you can disable the internal relay so constant power is supplied to the bulb, while the on and off commands are forwarded to your hub. You can then write an automation to turn on or off the bulb when the switch is activated!

I have a few different switches in my house after months of research to what would have simple on/off controls, but give me the most flexibility with progressive enhancements. Here are some of the switches that I currently use and the enhancements I get from them:

ecobee Switch+: This switch provides many benefits like a light (path lighting and night light), speaker (voice announcements, music, home alarm), microphone (control devices), illuminance sensor (to conserve energy), motion detection (security, lighting control), and temperature sensor. I integrated this switch into Home Assistant with the HomeKit integration.
GE Z-Wave Motion Switch: This switch mainly provides motion detection and works with Z-Wave. I use it mainly for motion detection. I do wish it exposed its internal light sensor so I could get access to the rooms brightness for additional smartness. This switch has been pretty reliable but every smart switch has its baggage too.

I have purchased and but have not installed the Inovelli Smart Switch. I plan on replacing my GE Z-Wave switch in the house to control the garage lights. This will allow me to use the switch's awesome LED lights on the side of the switch to tell me the current charging status of the Tesla Model 3.

Also, keep in mind that some switches may look nice and have full touch screens but these probably won't last as long as a normal switch and many people might not be able to figure out how to use them. Will you have to explain how to turn on a light to your grandparents or parents?

Lastly, here is a list of some of the automations I've used to improve accessibility:

Use the switch's motion sensors to automatically turn on the lights and off when there is no more activity. I've tried to be really smart about this, so one never even has to speak or touch a switch, it is not perfect but getting closer.
Use the switch's motion sensors and speaker (or nearby speaker) to announce switch state changes like (light turning on, motion detected, fireplace turning on). This is behind an automation accessibility mode flag and is useful to hear the state changes or what's going on in the room (and debugging).
Automatically turn on lights in the case of emergency (path way lighting).

Audio

TLDR: Think about ways to use both microphones and speakers with natural language to augment the home. There's more to it than just whole home audio. Your home can give context in scenarios where someone can't see or touch.

When end users typically think about about about smart homes and audio devices, they think about whole home audio, but this is not the only use case as I'll cover below.

Since whole home audio is all the rage and the biggest use case it is worth addressing. I feel that it is only possible for the large majority today within two ecosystems: Amazon via Echo (or third party speakers like the Sonos) and Apple via HomePod. Both of these allow you to use voice activation to play music and have deep application support. But they require an internet connection for streaming audio and cost some serious money. Smart switches like the ecobee Switch+ mentioned above can play music, but only on a single device at a time.

One accessibility item that people forget about is natural language commands. If you are interacting with your smart home using voice, than make sure various commands work like Turn on the office lights, Turn on the lights and Lumos. This second command is tricky, something I've yet to get working very good.

But you can do so much more with audio than just play music! Here is a sample list of automations I'm currently doing with audio:

Automatically mute devices based on time of day (privacy!).
Automatically normalize audio based on time of day, so a notification doesn't blare audio when someone is sleeping.
Use facial and image recognition to make contextual announcements like who is at the door (person with suggested age, glasses, etc), package delivered, cat is back from its adventures!
Use the switch's motion sensors and speaker (or nearby speaker) to announce switch state changes like (light turning on, motion detected, fireplace turning on). This is behind an automation accessibility mode flag and is useful to hear the state changes or what's going on in the room (and debugging).
Announcements when tasks are completed (e.g., the washer or dryer has finished).
Whole house notifications for announcements like dinner is ready or it is bed time. This can be a little annoying for switches which might have audio delivery delays as audio echos throughout the house.

Future automation goals relating to audio:

Use directional audio notifications with sensor/location data to point someone to the closest door in case of emergency.
Use speakers to amplify security alarms and doorbells.
Integrate with live weather advisories and keep informed about dangerous weather conditions.
Notifications when wildlife is outside (e.g., deer, ducks, etc.).
Clear alexa history.
Play relaxing music, white noise or lullabies at bed time based on occupancy.

If you are using Amazon Echo devices and are using Home Assistant, I'd highly recommend installing the Alexa Media Player Integration. I use this to control all of my devices as well as do text to speech. Here is the Home Assistant service call needed to send a message:

service: notify.alexa_media
data:
  message: "my message"
  data:
    type: "tts"
  target: ["media_player.YOUR_MEDIA_PLAYER"]

I created a reusable Node-Red Subflow that calls this service and has input_boolean and overrides for announcements.

Lighting

TLDR: There's more to lighting than just smart bulbs. Removing the need to turn on or off a single light is a huge deal for those with little to no mobility.

Lighting can be used to transmit vital information about current or future state. But the most often use case is to use color to control room ambience or enhance a movie experience through smart bulbs and hub. The current two most popular lighting solutions Hue and LIFX.

But you don't need smart bulbs to have great lighting. While I do have some smart bulbs primarily for my lamps, I've invested in smart switches to control my lighting. I feel like this gives me way more control over the environment in a future proof way that works with any lighting source, it also allows me to control lights by the switch, motion, voice or application. Which is a huge win as smart bulbs typically don't provide bright light and can lock you in on controlling them. With this said, I do think that smart bulbs and devices with integrated lights (switches, smoke detectors for pathway detection) can increase accessibility more than just a light switch.

Keep in mind, smart bulbs don't work with most switches and don't even think about using it with a dimmer switch.

One advantage of a smart bulb is that some can change colors. With color, you can show different states like danger, this way to safety, or work-in-progress. It is important to try and choose accessible colors that everyone can see!

Here is a sample list of automations I'm currently doing with lighting:

Turn on lights based motion, time of day, and outside and inside illuminance. It is really important to respect turning on or off the lights by manual press!
Turn on lights when people are detected in the driveway / door.
Turning off lights when the tv starts playing a movie.
Turning on lights and fans when shower turns on and off 30 minutes after shower stops.
Turn on/off devices and lights when waking up or going to sleep.

Future automation goals relating to lighting:

While my wired Nest Protect smoke detectors currently provide some basic path way lighting. I would like to have directional lighting to the exit based on occupancy. Think of bright light a head of you and fading light behind you.
Use a colored light to show busy status like in virtual meeting.
Show current traffic conditions for possible delays based on calendar location data.
Have greater presence detection so the home knows where you are at.

Other Devices

There are other many other device types, just keep in mind the devices responsibility and make sure it does it well. Progressive enhancements can be used to power future automations which can improve the quality of life for all.

Smart Home Software

I think about accessibility not just on physical devices but the software too. This could be the automations you wrote to the smartphone applications or management hub (e.g., Home Assistant).

We have a long way for software to be more inclusive from color schemes, easy clickable buttons, and screen reader support.

I'd like to see more work go into improving accessibility in Home Assistant (as well as this blog) as shown above. I think this will be improved over time which will be a huge win for the whole community.

Power failures

Lastly, what about power failures? Can we handle them in an accessible way where interaction isn't needed like automatic power fail over? For example do we handle the following scenarios:

Restoring to last state on power restart.
Running critical devices like smoke detectors, wifi on batteries.
Critical alerts like security, fridge and flood sensors while power is out.
Letting others know we might need assistance.

Conclusion

I feel those that have the most to gain with a smart home are those with disabilities and we need to make it easier. But, it feels almost impossible to install, configure, and automate a smart home today. There are just too many blocking factors such as:

Too many inaccessible product options and software.
Limited knowledgeable professional installers.
Steep learning curve to get a hub up and running, and integrate your devices.
Steep Learning curve learning how to write automations, especially for the majority who don't know how to program.
Hard to have reliable and secure networks (Z-Wave, Zigbee, Wifi). Having a network just for Internet of Things (IoT), is only available on high end routers (e.g. Ubiquity UniFi Dream Machine).
Consumers need to be privacy aware in a time when paid products are compromising user privacy for profit.

We need to take the steps to make Home Automation and Smart Homes more accessible in this early adoption stage of Smart Homes.

Also, be sure to check out volunteer opportunities like Be My Eyes for bringing sight to blind and low-vision people!

Lessons learned from working at home for over a decade

Blake Niemyjski — Tue, 10 Mar 2020 00:00:00 GMT

With the recent world events causing many to work from home, I figured what better time than now to share my some of my tips. This may be your first time working from home for an extended period of time, but I have been remote for the past 13 years.

1. Start with a routine

Whether you are a morning person or not, get into a routine that works for you and stick to it. This will help get you motivated for the day. I try my best to go to bed at a reasonable hour so I can naturally wake up around 5/6 a.m. feeling refreshed. I start my day enjoying two cups of organic coffee with Organic Valley Heavy Whipping Cream and Unflavored Collagen Peptides while catching up on emails, blogs and world news. This really helps me wake up and get ready for the work day.

I've had my trusty Keurig B60 for a decade or longer. I've been waiting to find a good coffee/mocha machine that is easy to clean and can be automated. If you have a machine that works for you let me know.

A routine could be as simple as taking a shower, getting dressed for the day or even going for a walk.

2. Create a daily goal

I start every day with a simple goal of what I want to accomplish. It can be something I want to learn or a work task I want to get done like:

Complete a feature request or comment on an issue
Inbox zero
Meditating for 10 minutes during lunch using Calm
Reading a page of a book

Just create a small goal that is accomplishable as it will make you feel more productive.

3. Work asynchronously

If possible, try working asynchronously. This means that you prioritize smaller similar tasks and complete them without interruption. Unless a message is super important it's okay to respond to it a few minutes later along with any other messages you get. It's proven that switching tasks is not productive.

If you have a hard time with time management, there are techniques like Pomodoro which may help you stay focused.

4. Keep your calendar up-to-date

If you have a morning standup meeting (e.g., scrum) pay attention for meetings or demos that may not have been sent via a calendar invite. I always add these to a calendar with any applicable time zones. Not doing so may mean you miss an important discussion.

If you keep to a schedule and know when upcoming meetings are you might be able to take a flexible lunch hour to get a haircut, mow the lawn before it rains, or have a quick lunch with a friend.

I also keep an eye out for coworker travel plans to the office. This allows you to spend time with other remote employees with whom you communicate with daily. It's super important to stay social even if that's not your personality! I cherish all the memories of visiting the office and having fun with those who you work with day to day.

5. Avoid burn out

It's very easy to get burned out by working super long hours to make up for what might feel like an unproductive day or to get a task done. Just consider the fact that working in an office may have more distractions than working from home. Making it a super productive day even if it doesn't feel like it.

There will always be a never ending supply of work, that will be there tomorrow. So leave work at work and disconnect to enjoy family/personal time at the end of the day. It really helps if you have an isolated space like a spare room where you only work from. At the end of the day close the door and walk away.

6. All about the Optics

If you don't have a camera and will be working from home for a while, I'd recommend investing in a good Logitech BRIO camera and Logitech H570e headset.

Also, make sure to check what is in your background of your camera and mute yourself when not talking.

I use the built in Apple MacBook Pro front facing camera and the Apple AirPods.

Just remember to look directly into the camera and smile ??. It goes a long way to build relationships and show that you are listening! Meetings don't always go smoothly and you are not the only one who may be experiencing technical difficulties.

7. Take a break

It is perfectly normal to take a break after working a few hours, diagnosing a hard problem, or you just need some fresh air. I find the most difficult problems are resolved upon returning from a short break. It's like having a second pair of eyes on a difficult problem.

Get up and take a walk to the mailbox or around the block to clear the mind (and pie in moderation never hurts).

</iframe>

Every one needs a few minutes of me time to decompress!

8. Exercise your body, eyes, and mind

It's really important to stretch and get your exercise in after being sedentary. I purchased a standing desk to help get me more active and I've seen others use under desk treadmills too.

I've always been told by my eye doctor to take breaks and to look outside in the distance. It's not good for our eyes to constantly focus on screens that are close to us.

It really helps if you have an amazing office view! It's amazing what you'll see when you take a look outside. In the video below I saw two male ducks fighting for love one summer day (looks best full screen @ 4k, tune to 2:30 for an up close fight).

</iframe>

Did you know that ducks form seasonal bonds?

Working out, meditating, and eating clean really helps keep your mind sharp too! You have the added advantage of removing food temptations from your diet by working at home!

9. Set family expectations

A lot has changed since I first started working remote in college. It can be difficult for a partner or family member who has never worked remote before to understand the work life dynamics. Although you might be home, you are not free all the time for home tasks. I'm guilty of forgetting to switch the laundry during a break because I'm so concentrated in my work.

If you have dedicated work hours stick to them. If you have flexible hours, than consider starting your day early so you can get tasks done that may need to be done before the weekend starts. This may include mowing the lawn or getting groceries.

10. Distractions

It can be easy to get distracted by the smallest of things! I'd recommend eliminating as many distractions as possible such as a messy desk or enabling screen time on your phone. Just make your environment comfy to be in for long periods of time without added distractions.

I've found that distractions can change day to day, so try and learn what triggers you. For example, one day I might be able to concentrate while playing music while others I need it to be dead silent.

Here are two music playlists that oddly enough help me concentrate:

Try narrowing down what music you can play at a reduced volume that doesn't distract you in your day to day tasks. I've automated my speakers to change the sound levels during the day so I don't wake a sleeping bear ??.

I've taken this a step further when I'm having a really productive day. I will start diffusing a specific scent (like mental focus). Scents can trigger memories and emotions which I find help me get back into a productive state of mind. The human body is truly amazing.

Lastly, it can really help to work with a coworker (e.g., pair program) to do a task if you are struggling (we all have these days). This helps ensure active participation while working through complex problems.

</iframe>

This doesn't mean that you will be twice as productive, but you will be more productive than just yourself!

11. With Great Responsibility...

I absolutely love working from home in my own personal space. But one regret I do have is not taking full advantage of working remote by leaving my house more often. For example, I wish in the past decade that I:

Traveled to more remote destinations while working. In 2016, my wife and I traveled and worked remotely in New Zealand for a month. This was an amazing learning adventure! I'd wake up early and work for several hours at the start and end of the day. It was a blast experiencing different cultures and breath taking scenery, but it takes extreme discipline.
Taken a cross country sight seeing train while working through the scenic landscapes.
Spent more time working from my parents who live a few hours away and are not getting any younger.

Installing Home Assistant on Ubuntu

Blake Niemyjski — Mon, 09 Mar 2020 00:00:00 GMT

NOTE: This post has been updated on May 26th to use Docker-CE instead of Snap.

Home Assistant has stated they will no longer support this installation method but there currently is no other way to use external storage.

I recently upgraded my Home Assistant server to an Intel NUC 10 Performance Kit (NUC10i7FNH) from an Atomic Pi. I needed to manually install and configure Ubuntu on my hardware there is an issue where you cannot boot from the NVMe drive. and I wanted more control over secondary storage and services.

If you are looking for hardware recommendations to run Home Assistant or a small server on please check out Upgrading Home Assistant Hardware.

Updating Firmware

It is always a good idea to update to the latest firmware of any device you purchase before starting to install the Operating System (OS). Firmware upgrades can add additional features, hardware support, or fix bugs that could cause installation failures.

I started by navigating to the Intel download center and searching for my model number (NUC10i7FNH). I clicked around until I found a page talking about available Bios updates. In my case it was called BIOS Update and Recovery Instructions and contained detailed instructions on how to update the Bios.

Operating System

If you are only using Home Assistant and don't plan on running any other services on your machine than I'd recommend just using Home Assistant formally known as hassio. This will be automatically updated and managed for you, reducing your maintenance which is always a good thing!

Since writing this article, Home Assistant has dropped support for Ubuntu and only supports Debian. I HIGHLY recommended to use Home Assistant OS.

However, if you run want to run extra services on your machine like facial/image recognition software you'll need to install Home Assistant on an existing Operating System (OS). Additionally, there may be software or driver limitations with the Home Assistant images like the issue with not being able to boot from the NVMe drive that will force you to install on an existing OS.

I use the Ubuntu Server LTS server images instead of Windows for the following reasons:

The software is open source, free and used by many.
Active community support.
Ubuntu Server is light weight and works on many different hardware architectures (e.g., x86, arm).
LTS images are supported for long period of times and updating is a breeze.
Documented installation steps for Home Assistant.

After you download latest Ubuntu Server LTS from the Ubuntu site. We need to image an MicroSD card or USB Flash Drive) with the downloaded image. I use Etcher to format the drive with the downloaded Ubuntu image (iso).

Install

After inserting the imaged flash drive or MicroSD card into the computer, I follow the standard install process for Ubuntu. I use the recommended settings for formatting and partitioning the primary NVMe drive.

I could also format the secondary drive during this step. But I always tend to just format the secondary drive later as I can then immediately configure the mount points after formatting the secondary drive.

I always install and configure SSH when prompted. This makes it so much easier to complete the install below using your main computer so make sure to import your SSH identities when prompted.

After the installer reboots the machine, you can ssh IP_ADDRESS into the machine and use your primary computer to finish the install. Please remember to replace IP_ADDRESS with the machines IP Address. You can get the IP Address by resolving the hostname you gave the machine during setup, looking in your router settings, or by running the following command:

hostname -I

Just grab the first IP Address in the list of results.

You can also continue the steps below directly on the machine if you didn't configure or know how to use SSH. Here is a beginners guide to SSH.

Next, the steps below require sudo to execute privileged statements from the terminal.

sudo -s

Optional: Enable Canonical Livepatch Service

The Livepatch Service allows you to apply critical kernel patches without rebooting. Which lowers the amount of time maintaining my install. You just need to get a free Livepatch key and enter in the commands below (remember to replace KEY with your Livepatch key).

snap install canonical-livepatch
canonical-livepatch enable KEY

Optional: Format and mount secondary storage

I always install a secondary storage device to place my backups and other data on. I feel like this helps protect me from boot drive failures and makes my life easier as all I'd need to do is replace or reformat the failed primary drive and I'd still have access to my backups locally.

The following steps work for any storage devices (e.g., SSD or MicroSD). I feel like every time I need to format a drive in linux it's been so long that I forget how to and then I spend forever figuring out how to do it. This time around I came across this awesome guide which I'll follow below with some tweaks.

I want to access the drive by a friendly name via /mnt/data after it's been formatted. For this to happen we first need to create that directory so in the following steps we can mount our storage device to this folder.

root@nuc:~# cd /mnt
root@nuc:/mnt# mkdir data

Next, we need to get the location of the new drive by executing the statement below and looking for the logical name which in my case is /dev/sda.

root@nuc:~# lshw -C disk
  *-disk
       description: ATA Disk
       product: Samsung SSD 860
       physical id: 0.0.0
       bus info: scsi@2:0.0.0
       logical name: /dev/sda

Next, we need to create the GUID Partition Table (GPT) and enter in Yes to the prompts when asked.

root@nuc:~# parted /dev/sda mklabel gpt

Next, we need to create the partition label

root@nuc:~# parted --align optimal /dev/sda mkpart primary ext4 0% 100%

Next, we need to create the partition

root@nuc:~# mkfs.ext4 /dev/sda1

Next, we need to get the drives UUID identifer so we can safely mount it. If you have a few in the list, we are looking only for our drive we've identified earlier (e.g., sda1).

root@nuc:~# ls -l /dev/disk/by-uuid/
total 0
lrwxrwxrwx 1 root root 10 Mar 9 02:40 87f48668-003f-42b2-b9b9-d588093c7d22 -> ../../sda1

Next, copy the UUID from the previous command (in my case 87f48668-003f-42b2-b9b9-d588093c7d22). We'll need we need to update fstab to mount our device on boot.

root@nuc:~# nano /etc/fstab

Once this text editor is open, navigate to the end of the file and add the following line replacing my UUID f58e18af-1fb2-4186-92f6-702c081a757e with your own.

/dev/disk/by-uuid/f58e18af-1fb2-4186-92f6-702c081a757e /mnt/data ext4 defaults 0 0

Once this has been added press the control+o, then press enter key to confirm and finally control+x to exit nano.

Next, we can remount everything defined in /etc/fstab without rebooting by running the statement below.

root@nuc:~# mount -a

Something that I struggled with in previous attempts was knowing what device a folder was stored on. I came across this awesome little command df that can confirm that /mnt/data is located on the correct storage device (/dev/sda).

root@nuc:~# df /mnt
Filesystem     1K-blocks     Used Available Use% Mounted on
/dev/nvme0n1p2 959862832 10726424 900308248   2% /
root@nuc:~# df /mnt/data
Filesystem     1K-blocks  Used Available Use% Mounted on
/dev/sda1      960379920 77852 911447644   1% /mnt/data

As you can see the parent folder /mnt is on my primary NVMe drive, while /mnt/data is located on our recently configured SSD drive (/dev/sda)!

Optional: Place backups on secondary storage

If you don't have a second drive already configured. Please install one and follow the steps above for formatting and mounting secondary storage before continuing.

NOTE: You can allow store the addons folder on a secondary drive following the same steps below but replace backup with addons.

We need to create a our backup directories for Home Assistant by running the commands below.

cd /mnt/data
root@nuc:~# mkdir /mnt/data/backup
root@nuc:~# mkdir /usr/share/hassio
root@nuc:~# mkdir /usr/share/hassio/backup

Next, we'll need we need to update fstab to automatically mount our backup folder on boot.

root@nuc:~# nano /etc/fstab

Once this text editor is open, navigate to the end of the file and add the following line

/mnt/data/backup /usr/share/hassio/backup none defaults,bind 0 0

Once this has been added press the control+o, then press enter key to confirm and finally control+x to exit nano.

Finally, we can remount everything defined in /etc/fstab without rebooting by running the statement below.

root@nuc:~# mount -a

Installing Home Assistant

This step installs Home Assistant following the Generic Linux Host Guide.

apt-get update && apt-get upgrade
apt-get install -y apparmor-utils apt-transport-https avahi-daemon ca-certificates jq curl dbus network-manager socat software-properties-common
systemctl disable ModemManager
apt-get purge modemmanager

Next, follow the installation documentation for installing Docker-CE and then run the scripts below to finish installing Home Assistant.

curl -sL https://raw.githubusercontent.com/home-assistant/supervised-installer/c674830d8ddc6af9d618755a7995af939dd73fde/installer.sh | bash -s

It might take a minute or two for Home Assistant to get up and running. It has to download all of the docker containers from the docker registry and then run the containers.

If you want to know more about docker I'd recommend watching this video.

You can access your Home Assistant install by navigating to http://IP_ADDRESS:8123 in your browser of choice. Please remember to replace IP_ADDRESS with the machines IP Address. You can get the IP Address by running the following command:

hostname -I

Just grab the first IP Address in the list of results.

It's highly recommended that you configure your router to give this machine a static IP Address. This ensures that the IP Address doesn't change when the machine reboots.

Optional: Restore from previous backup

The first step is to create and download a backup from your previous install. Once this is done we need to transfer it to the new machine we just configured.

Create a new temporary Home Assistant account if you haven't done so already and install the Samba share official Home Assistant add-on by navigating to http://IP_ADDRESS:8123/hassio/addon/core_samba.

Next, configure a password in configuration and click the Save button. Then click the Start button to turn on the Samba add-on.

Once it's up and running you need to connect to the Samba share and upload the downloaded backup to the backups folder.

If you've never connected to a Samba share before then please watch one of the following videos:

How to connect to Samba share on Windows 10

How to connect to Samba share on macOS

Next, navigate to the http://IP_ADDRESS:8123/hassio/snapshots snapshots page and click the refresh icon in the upper right. The snapshot will be listed in the available snapshots. Click on the snapshot and then selected Wipe & Restore.

After a few minutes home assistant will restart and you'll have all of your existing configuration and addons from your previous install!

Optional: Configure Z-Wave Integration

If you are using Z-Wave, you'll also want to update your Z-Wave configuration usb_path. If you haven't done so already plugin your Z-Wave controller (e.g., Aeotec Z-Stick Gen5) into any usb port and run the following command to get the USB UUID.

root@nuc:~# ls /dev/serial/by-id/
usb-0658_0200-if00

The command returned usb-0658_0200-if00 a UUID for my USB device.

Next, we need go and update the Z-Wave configuration yaml using your favorite editor.

zwave:
  usb_path: /dev/serial/by-id/usb-0658_0200-if00

If you configured Z-Wave using the integrations page, you may need to remove and readd the integration in order to set the USB Path.

Finally, save the configuration file and restart Home Assistant to pick up the new configuration settings.

Upgrading Home Assistant hardware in 2020

Blake Niemyjski — Tue, 03 Mar 2020 00:00:00 GMT

I've upgraded or swapped my Home Assistant hardware more times than I want to count. I want to share the experiences I've learned along the way when choosing hardware to run Home Assistant locally. I always try choose hardware is lower power and will take up limited space in my 9U Server Cabinet

Skip to March 2020 for a summary of what I recently upgraded to and the Conclusion if you are looking for current hardware recommendations.

July 2018

In early 2018, I spent a massive amount of time researching what hardware and software I wanted to use for my smart home. My requirements were to use hardware that I could repurpose if needed and open source home automation software that was widely supported, low maintenance, low power and cheap. The Raspberry Pi 3 B+ was the clear winner at the time with Home Assistant meeting the other requirement.

Hardware	Cost
Raspberry Pi 3 B+	$38
Samsung PRO Endurance 64GB	$18
Miuzei Case	$17
Total	$73

I followed this guide to install Home Assistant on the Pi. It was fully manged and worked great as I didn't want to have to manage an Operating System too.

By early 2019, I was looking for something new for multiple reasons:

I had multiple drive failures due to high number of writes going to a basic MicroSD card that wasn't designed to handle the number of Home Assistant writes for sensor data. I upgraded to the Samsung PRO Endurance 64GB MicroSD card which helped solve this problem.
Any power interruptions could cause an SD Card to be unbootable. This can happen if a write is happening to the SD Card when the power loss occurs. I had to restore from a saved backup multiple times which is a real pain and not something you look forward to.
Too many docker add-ons and integrations running was causing resource constraint issues where integrations were slow to respond. This was greatly magnified anytime an integration that required a network connection was offline due to configured long default timeout times in software and not handling backpressure.
Rebooting for updates or integrations was painful.
The acceptance factor was dropping.

When purchasing flash memory it's highly recommended to choose storage that is optimized for throughput (reads/writes) as well as total number of writes (TBW).

April 2019

One April morning I was super excited to see an announcement for a new Single Board Computer (SBC) that had

16GB of onboard Flash Memory. This solved the issue with power interruptions.
2GB of ram. Not only was it faster, but it was twice the amount of memory compared to the 3B+.
Faster x86 hardware which meant that it could out preform the Pi in every way as well as run add-ons like VS Code (which didn't yet support ARM).
Runs on only 4-15 watts TDP! Didn't require too much more power to run.

Hardware	Cost
Atomic Pi	$36
Power Supply	$10
Reused Samsung PRO Endurance 64GB	-
3D Printed Case	-
Total	$46

This was a clear winner and checked all my boxes but it has some definite pain points like:

No fully managed Home Assistant Images from Home Assistant.
The only "official" device images were bloated developer OS Images. I tend to only install clean images from Ubuntu or Microsoft directly so I know I'm getting a clean experience free from any "paid" vendor software. That's not to say these images weren't already clean. I just want to know what I'm using.
No documentation on how to install Home Assistant on it other than this.
Some Atomic Pi's might have issues with it requiring a hard power off for reboot. I found this out when I helped get my neighbor up and running on an Atomic Pi.
The primary 16GB drive is an eMMC which is known to have limited write lifespan. It's fine for putting an operating system on but not something that is always writing to it. So you need to configure Home Assistant to write to the MicroSD card. It could be corrupted but it's much much less likely to cause issues. In fact, I've had 5 more power outages than I'd like and I didn't run into any issues.
You have to know and maintain the linux install.

I was also able to repurpose the Raspberry Pi 3 B+! I installed OctoPrint on it and it controls my Ender 3 Pro.

By early 2020, I was starting to hit some of the same issues I ran into with the Atomic Pi:

The addons and integrations I was running was just too much for the hardware constraints. I currently have over 800 entities in my House and Garage with many sensors including Facial Recognition which is all happening locally. When backpressure occurred with networked entities on top of all of my local processing... I started to get the feeling that some of my Node-RED automations weren't running.
Software reboots for updates or adding new integrations takes too long.

With these issues I was on the lookout for a new SBC that had much more processing power while still had a low TDP power envelope. I knew I still had quite a bit of life in the Atomic PI but long term this was not the solution for me.

As for the Atomic Pi, I'm thinking of running some analysis services on it. It would be really cool to buy two more and make a Kubernetes cluster! What are your thoughts?

March 2020

I started researching the next hardware that would replace the Atomic Pi. I wanted to take a look at all options since this next choice I would be living with for as long as possible.

I could have continued to use with the Atomic Pi for quite a while longer. Especially if time was taken to profile and optimize automations and integrations. As the saying goes time is money and I think a lot of time would be sunk in just this.

Since the Atomic Pi came out there was a new Raspberry Pi 4 Model B released. I knew it would roughly be the same experience as the Atomic PI, but impossible to compare benchmarks due to different CPU architectures. From the benchmarks I've seen (not comparing apples to apples) I knew it would be close. It would still have the same storage issues as described previously and it might still have USB hardware issues with the Aeotec Z-Stick. I need something more powerful that will last for years to come.

I was looking for AMD to release a Ryzen 4000 series embedded device but this just doesn't seem to be happening and it's very unlikely one can build a super low powered tiny desktop SBC any time soon. The Embedded Ryzen series just don't seem like it will cut it in processing power either which is a real shame.

This lead me to look at Intel's latest 10th Generation Intel NUCs (NUC 10) that were released in December of 2019. The processors are built on 14nm++, are faster and also contain hardware security fixes make it worth considering over previous generations (NUC 6, NUC 8). One cravat is that the NUC 10 actually has a slower graphics cards (GPU) than previous generations. This should only affect you if you are wanting to use the integrated card for heavy machine learning or other compute operation. I really had hoped it was a bit more powerful but it will work.

The 10th Generation met all my needs (6 CPU Cores @ 28 watt TDP) but I wanted to be a bit more future proof with newer standards. I initially ruled out the NUC 10 for these reasons and was waiting to upgrade to the 11th Generation Tiger Lake NUC later this year. Tiger Lake is supposed to feature 10nm technology, faster DDR4 memory, PCI Express 4.0, HDMI 2.1 and a new much faster Xe GPU. Also, I know that they are coming out with a version that will take any PCI Express x16 graphics card. This would be really awesome for machine learning locally. I'm just cringing what the price tag and power consumption of such a rig would be.

TIP: Check out this awesome Intel NUC tracking spreadsheet if you want to stay up-to-date on what NUCs are available.

But this all changed, last week when there was an awesome deal on an NUC 10 (NUC10i7FNH) kit with 64GB of memory included. A few of us were lucky enough to snag it at $605 USD.

Hardware	Cost
Intel NUC 10 Performance Kit (NUC10i7FNH)	$605
Samsung 970 EVO Plus 1TB - M.2 NVMe	$220
Samsung 860 EVO 1TB - SSD	$155
Total	$980

Compared to the previous two solutions this is definitely the most expensive and a bit overkill. I went with two drives because I want the 860 EVO for backups as well as large data sets facial/image recognition. I could have saved $179 on by going with 500GB drives instead. I realize that I might not ever use this amount of storage, but I don't want to run out. I can also use this memory in a future machine or some of it for network attached storage (NAS).

The current downsides of the NUC 10 are as follows:

Cost
Higher power usage at 28 watt TDP (can peak much higher for short duration).
Unable to use the Intel-NUC system images as they currently don't support (as a result you have to maintain a linux install):
- Unable to boot from the NVMe drive
- Limited driver support
- To my knowledge you cannot attach secondary drives for storage.

I should have spent some time trying to install it on my NUC to see if there were any other issues or if I'd run into driver issues but I was quite turned off by the lack of NVMe support and couldn't get any answer on the official discord channel on secondary drives.

So far the NUC 10 has been working amazing and everything is so much faster and working much better than the previous hardware!

Conclusion

If you are looking for a SBC/small machine to run Home Assistant on I think there are some great options out there. Even though I choose an Intel NUC 10 series doesn't mean you have to (I have some insane requirements). I'd recommend starting small if you are just starting out. Below are my recommendations (based on the machines I have used) if you are looking for a machine today.

NOTE: If you don't care about power usage or form factor you can build a machine on the cheap. I would build based on the Ryzen 4000 series when it is released.

There are also many other small form factor machines available that are much cheaper. But the technology stack may contain hardware vulnerabilities, may no longer be supported, or may lack community adoption/support. So please keep this in mind when picking a solution.

Budget $75 USD

I really like the Raspberry Pi 4 Model B. But I cannot recommend it until the USB hardware issues with the Aeotec Z-Stick are worked out. Also, I'd only consider it if you have a power backup solution so you don't run into any power failure issues described above. It has cases you can buy from any retailer and you can use the managed system images.

Due to the hardware issues that have been present since launch (some have been resolved), I can only recommend the Atomic Pi which is the same exact cost but features onboard flash storage and is x86 hardware. The main con being you have to install and manage linux.

Atomic Pi

Hardware	Cost
Atomic Pi	$36
Power Supply	$10
Samsung PRO Endurance 64GB	$18
3D Printed Case	-
Total	$64

Budget $575 USD

I know there is a big price jump up from the previous tier. I just feel like it is not even worth considering a non quad core cheaper solution like the i3 NUC. In my experiences I've ran into to many issues with lack of processing power.

If you can't hold out until the Tiger Lake NUC is released I'd buy an Intel NUC 10 with only a single SSD (as you can't boot off the NVMe drive). You could then use the built in system image and let it manage everything for you.

As of writing this the NUC 10 (NUC10i5FNH) seems to have very limited stock so I listed the NUC 8.

Hardware	Cost
Intel NUC 8 Mainstream Kit (NUC8i5BEH)	$439
Corsair Memory Kit 16GB DDR4 2400MHz	$65
Samsung 860 EVO 250GB - SSD	$60
Total	$564

Budget $850 USD

If you can't hold out until the Tiger Lake NUC is released I'd buy an Intel NUC 10 with a backup drive. Here is a recent review by AnandTech

Hardware	Cost
Intel NUC 10 Performance Kit (NUC10i7FNH)	$605
Samsung 32GB DDR4 2666MHz RAM	$122
Samsung 970 EVO Plus 500GB - M.2 NVMe	$116
Total	$843

You could bring the price down by swapping out different components but I feel this offers you the biggest bang for your buck and leaves you open to future upgrades via additional memory and storage.

Please stay tuned for my next guide on installing [Home Assistant on Ubuntu]](/blog/installing-home-assistant-on-ubuntu).

Thinking about smart home power usage

Blake Niemyjski — Thu, 13 Feb 2020 00:00:00 GMT

I've spent a great deal of my life admiring nature and thinking of ways we can do to reduce our footprint. I'd like to think I've taken steps both physically and monetarily to do so.

The Dreaded Power Bill

Which is why one Wisconsin Winter day I opened my utility (gas and electric) bill and had a not so pleasant moment. Complete sticker shock at the $410 USD single month total, which was our highest bill yet. I almost immediately thought this can't be right, only to get the power bill break down only to see some mistakes had been.

Power Bill Insights

I analyzed the electric bill by looking at the kWh break down. This was telling me a couple things:

The Telsa Model 3 was starting to charge every night at 10:00pm. This is exactly when time of use enters the cheapest rate (or so I thought)!
I was not entered into Three-Tier Time-of-Use program like I had previously thought. Time-of-Use (TOU) billing allows you to have cheaper or more expensive billing periods based on historic grid usage.

Time of use rates are a type of electricity rate used by some utilities for billing customers. Under a TOU rate, customers pay different prices per kilowatt hour (kWh) of electricity that they use, depending on when they use it. Pricing varies by time of day, and can also vary based on the day of the week (weekend or weekday) and the time of year (Aurora).

I thought I had enabled this eight months previously by entering into the time of day program website. However the opt requests must go straightly to the garbage, because it was never processed. It just took a call to the power company to get it enabled. I learned a valuable lesson to always follow up to ensure things get done!
Some power hourly usage spikes looked like they could be caused by something running on a schedule. I immediately started thinking about the thermostat comfort settings.
The max kWh usage excluding Tesla charging seemed like average use.

This lead me to reexamine what I was doing as well as look into products that might help me determine what is consuming electricity.

Measuring

I was talking to one of my awesome neighbors about how our bill had gone way up from the previous month. I was happy to know I wasn't alone and there are tools that can help. I was lent a Kill A Watt Electric Monitor to help measure kWh usage by devices that are plugged in.

I had never seen one of these before and it blew my mind that this existed. It monitors the electricity usage of the device plugged into it and reports many facts like price (per hour, day, week, month, year) based on single use rate.

Advice: The longer you measure the more accurate your results will be!

This really helped me track down what my fridge, freezer, 3D Printer, smart device (Alexa, Gaming Consoles), and server rack were costing per month. It was eye opening to see the the Amazon Echo 3rd Gen speaker was costing me a few dollars per month to run (calculated using $0.10 per kWh). The real shocker was when I decided to measure only the Wemo mini smart plug with nothing connected was costing me $0.31/day.

I do want to remeasure this to verify as it does seem a bit excessive on the usage front.

I luckily already upgraded every single light bulb to the lowest wattage LED bulb while maintaining bright white light! If you haven't done this, then I recommend to calculate light bulb costs per room and keep this in back of your mind. It will help you purchase efficient LED bulbs and think about how to optimize the power usage of each room.

Residential LEDs -- especially ENERGY STAR rated products -- use at least 75% less energy, and last 25 times longer, than incandescent lighting (energy.gov).

The one thing I haven't figured out is how much energy is my hard wired smart devices costing me (e.g., smart switches)? I was betting that if I looked at the product details it would be unlikely to tell me how much it costs to operate! So was there already a monitoring solution that could help me?

Near Real Time Data From Power Company

Last year my utilities all upgraded to smart meters. These meters report usage data wirelessly as well as via Zigbee. I really like this as it's a win for everyone! The utility companies don't have to drive around, the environment is less polluted by internal combustion engines (ICE), and end users can see near real time usage.

I was discussing solutions with Ryan Hoffman, and he showed me his power companies near real time usage using these meters and I was soo jealous.

This was exactly what I needed so I called once again the power company as well reached out on twitter (squeaky mouse gets the cheese) and asked how to get Zigbee read-only access. I was basically told there was no Zigbee hardware support on any of their devices and they had already evaluated this. I was upset to say the least as I had already read the meters user manual and knew this was not true. After some frustration and explaining how they were incorrectly informed... I was eventually called back by a meter tech who had researched this exact scenario. Long story short, there are some limitations needed to be worked through to enable it for everyone and it's still a ways out.

Energy Monitoring Device

When I learned that I can't use the power companies near real time data, there had to be a way to get it from inside of the house. I started searching for solutions and came across a lot of solutions but not very many met my criteria:

Let me see my usage in near real time (eventual consistency :D)
Let me query the data so I can make informed decisions
Low maintenance
Easy to install
Work with existing devices and software.
Low power consumption of measuring device.
[Optional, but not happy about it] work offline

There were only two devices I've found that match all or most of this criteria.

The Aeotec Z-Wave Home Energy Meter uses Z-Wave to report usage to Home Assistant. I really like that it works completely offline and I have a friend that has had much success using this!
The Sense uses out dated 2.4 GHz WIFI 3 to report usage to the cloud. You can then pull this data into Home Assistant. I don't like that this requires an internet connection, but the pros are pretty unique. Sense can support detecting most devices (I've read it can have problems detecting LED devices) and can let you know when that device turns on or off. I like that it also supports Solar!

I've yet to purchase any solution yet, but I'm leaning towards the Aeotec. It is 1/3 the price of the Sense and it runs locally. I just hate giving up solar option and detecting devices. I will try and review the solution I pick in a future post.

Short Term Solutions

If your time is precious I would suggest starting in this order.

Communicate with the whole house hold about current usage patterns and how to find a happy medium.
Optimize your furnace schedules so more zones run at the same time and update the comfort settings to be warmer or colder depending on environment.
If your usage (e.g., charging a car) can be scheduled, then opt into Time-Of-Use based billing through your utility company. Also make sure to update your devices to optimize Time-Of-Use! I enrolled all of my ecobee thermostats into eco+.
Measure your usage and find the quick ways to reduce energy. Turn off things you are not using (e.g., the coffee maker).
Write automation routines or turn on settings that turns off bulbs when there is no motion/occupancy. Remember smart bulbs always require power.
Upgrade to LED bulbs.
Seal windows for winter and look into getting more insulation installed.

Long Term Solutions

I'm starting down this path today and here are the steps I'm currently planning on taking.

Better measuring of devices with a smart home energy metering solution.
Analyzing device power and user usage data and writing smart automations that reduce power usage without restricting limiting comfort. Some examples include:
- Run the pond air pump at off peak times or when people are observing the pond. I think this can be done with the Aeotec 40 amp smart switch
- Only turn on half of the room lights during high peak time (e.g., kitchen sink, a lamp).
- Advanced furnace zone scheduling learning off calendars and history.
- Better climate management by opening blinds when the sun is shining in on a cold day, but leaving it closed when it's hot out.
Buying lower energy smart devices. I'm starting to analyze everything I buy to ensure they are as efficient as possible. I'm currently running my smart home on an Atomic Pi which runs Home Assistant and much more using only 4-15 watts. I'm planning on upgrading it to a low powered AMD Ryzen 4000 machine or a 11th generation Intel NUC (15 watt TPD). I have been considering the 10th generation Intel NUC as well if I need to upgrade before the 11th generation devices arrive.
I want to install either Solar Panels and Batteries to help offset my power usage and ensure it's coming from a green source. I've been evaluating replacing my aging garage roof with a Solar Roof and a Power wall as users have indicated huge savings when using Time-Of-Use Billing! Here is my Tesla referral if you are thinking of doing this too!

Conclusion

I hope I gave you ways to measure and reduce consumption! Do everything you can to reduce and reuse in your life, leaving it better for future generations!

Hello Home Automation, How-to detect battery levels with Node-RED

Blake Niemyjski — Mon, 03 Feb 2020 00:00:00 GMT

I've decided to start off 2020 by converting my blog to Hugo and start blogging again! This is my first of many posts on home automation! This post is going to cover generically detecting battery levels with Home Assistant and Node-RED.

TLDR: Use the Function node defined in the Node-RED snippet here to properly handle all the edge cases.

I decided to share my experiences based on seeing what others were doing in this thread. For the most part, I saw users not accounting for edge cases or the Switch node wasn't going to work how they thought it was going to work.

Two of the main gotchas I've come across with device types that report battery state:

storing it in different locations (msg.state, msg.payload.attributes["Battery Level"], etc.)
differing values/data types (off, Charging, "100", 100)

This really makes it difficult to use the Switch or Change node with conditional logic as it might not behave how you think it does. This happened to me when I was debugging my first few attempts and getting 10 outputted in a 100 bucket via the gt condition. The best way I've found to handle these scenarios is with a Function node.

Home Assistant Feedback and Edge Cases

I think it may be worth opening a GitHub issue for this. It sure would be nice if devices that can charge have their charging state normalized, battery level state in the same spot with a last changed in attribute. Additionally it would be nice to see something for dead batteries or if the device just hasn't reported in a long time. I say this because I've have many devices I know are charged and on, but they are being reported as off. What do you think?

Examples

Off, clearly the battery has a level and it lasts 10 years and it's reported in the nest app as OK.

{"entity_id":"binary_sensor.nest_protect_battery_health","state":"off","attributes":{"friendly_name":"Nest Battery","device_class":"battery"},"last_changed":"2020-01-11T11:42:50.976581+00:00","last_updated":"2020-01-11T11:42:50.976581+00:00","context":{"id":"2a2f5c1503cd44c7bb3ebd9be69a4144","parent_id":null,"user_id":null},"timeSinceChangedMs":2731510}

Charging - battery level stored in attributes.

{"entity_id":"sensor.battery_state_2","state":"Charging","attributes":{"Battery Level":52,"friendly_name":"Battery State","icon":"mdi:battery-charging-40","device_class":"battery"},"last_changed":"2020-01-11T11:43:06.616903+00:00","last_updated":"2020-01-11T11:43:06.616903+00:00","context":{"id":"a9cc54cd33cf4cf1954c4cd2f2f633a8","parent_id":null,"user_id":null},"timeSinceChangedMs":2715885}

Same device as Charging example above but duplicate battery device sensor with state in the correct location.

{"entity_id":"sensor.battery_level_2","state":"52","attributes":{"Battery State":"Charging","unit_of_measurement":"%","friendly_name":"Battery Level","icon":"mdi:battery-charging-40","device_class":"battery"},"last_changed":"2020-01-11T11:43:06.599389+00:00","last_updated":"2020-01-11T11:43:06.599389+00:00","context":{"id":"60469c2964504dc4bb11cbf5a307ceb9","parent_id":null,"user_id":null},"timeSinceChangedMs":3766731}

August lock reported from august integration missing information.

{"entity_id":"binary_sensor.august_door_locked_battery_level","state":"off","attributes":{"friendly_name":"August Door Lock Battery Level","device_class":"battery"},"last_changed":"2020-01-11T11:42:50.892228+00:00","last_updated":"2020-01-11T11:42:50.892228+00:00","context":{"id":"567d9c771c91440c826ed26d14187513","parent_id":null,"user_id":null},"timeSinceChangedMs":2731591}

August lock as reported from Z-Wave.

{"entity_id":"sensor.august_asl_03_smart_lock_battery_level","state":"100","attributes":{"node_id":2,"value_index":0,"value_instance":1,"value_id":"72057594077773825","unit_of_measurement":"%","friendly_name":"August ASL-03 Smart Lock Battery Level","device_class":"battery"},"last_changed":"2020-01-11T11:47:39.195031+00:00","last_updated":"2020-01-11T11:47:39.195031+00:00","context":{"id":"406c8c33527a4daf8d9b23531953d54f","parent_id":null,"user_id":null},"timeSinceChangedMs":2443740}

Handling the edge cases

The best way to handle all the edge cases above is with a Function node. It will be extremely tough to handle normalizing the battery level with any other node type with out a really crazy series.

It should be everyone's goal to keep automations as simple as possible. This keeps the maintenance super low and the acceptance factor has high as possible!

So lets get started by creating a new Function node with 6 outputs.

// NOTE: We could probably look at the unit_of_measurement if specified to properly normalize the battery level.
// Check for battery level as it's set when state is Charging,Not Charging,etc...
const batteryLevelAttr = msg.payload.attributes && msg.payload.attributes["Battery Level"];
const state = !isNaN(batteryLevelAttr) ? batteryLevelAttr : msg.payload.state;
const level = parseInt(state);
if (!isFinite(level)) {
    return [msg];
} else if (level < 10) {
    return [null, msg];
} else if (level < 25) {
    return [null, null, msg];
} else if (level < 50) {
    return [null, null, null, msg];
} else if (level < 75) {
    return [null, null, null, null, msg];
} else {
    return [null, null, null, null, null, msg];
}

By creating 6 outputs and outputting the message to the desired output, we can wire up routines to fire on only the battery level ranges we care about (e.g., 0-10%, 10-25%, etc.)!

Bringing it all together

I've created a Node-RED flow to get every device that supports battery levels and output them to a Debug Node using the above Function Node.

You can grab the complete Node-RED snippet here.

Speaking at BAM

Blake Niemyjski — Tue, 19 Feb 2019 00:00:00 GMT

I'll be giving a talk at From BAM-Big Data, Advanced Analytics, Machine Learning Scaling Your Applications and Skill Set on January 15th!

Overview

Blake will lead a discussion regarding the struggles around starting and using open source projects to build software solutions for customers. Going from having no customers, to doing 3+ billion requests per month in Azure was a sometimes painful experience, especially on a limited budget. Blake will discuss what he has learned, as well as talk about data acquisition and maintenance, and talk about problems working with large data sets.

Speaking at Central Wisconsin It Conference

Blake Niemyjski — Thu, 04 Oct 2018 00:00:00 GMT

I'll be giving a talk at Central Wisconsin IT Conference on Exception Driven Development on October 6th!