I have decided to halt any plans to maintain the extension and focus on other spare time open source projects instead. I should have probably made this decision about seven months ago, when Twitch integration broke, however this extension means a lot to me. It was my first browser extension that still exists and went through a lot of transformations. All said and done, however, I don’t really have a personal need for it anymore and don’t like some of the changes I’d have to make to it to keep it working.

I’ve archived the GitHub repository and pushed out one last version that informs any remaining users of the demise of the extension (and possibly bringing you here). I will delist the extension a couple days after that update, once I’m satisfied it will have reached most people it could reach. The reason I’ve made the extension non-functional is so I can retire the API credentials used by the extension.

The website stays available at https://freaktechnik.github.io/justintv-stream-notifications but the streamnotifier.ch domain should no longer be considered safe or related to the extension.

I’ve pushed the “latest” in progress code to the next branch on GitHub. The plans for it were three major changes:

• Make actions and notifications modular. This means letting the user customize the actions available on channels and the primary action, as well as letting the user set up custom rules and patterns for notifications. This would remove a lot of rather brittle code.
• Build a new on-boarding experience that guides you through adding your first channel and optionally user
• Begrudgingly add a way to OAuth as a user of a provider. This would have to be mandatory for Twitch and would also improve reliability of YouTube

It is this last one that is a big reason for why I no longer feel enthusiastic about working on the extension. I would have to force users to bind their account identity to the requests the extension makes.

Apart from Twitch requiring authentication of a user for API requests, YouTube has also been a pain to work with. Not only is their API barely ready for live stream applications and the public rate limit is hit about half of the day, they also conduct approximately yearly audits of their API applications, which in principle is fine. They make sure applications using the YouTube API comply with their API terms. However, all of those audits have ended up with just a rapid fire of anything that looked like it could be a violation, leaving it to me to show them again and again how I am not violating their terms of use for the API. For the few correct violations that came out I’d usually get a deadline of a couple of months at best. This has lead to me half-heartedly building fixes that kind of made the extension compliant, with cleanup left for later.

I didn’t really want to invest a lot of time into them, because I was spending most of the time for the extension on planning out the actions/notifications rework. There were also a bunch of small maintenance tasks that kept piling up, like disabling the Smashcast and Mixer providers or updating dependencies. Further, as the extension got less important to me, I spent less and less time on it.

I’ve now decided that instead of every now and then spending some time on this extension I’d rather use my spare time to build extensions that empower users and make the internet better for small services and decentralized solutions. As such, my latest extension (which is also already a couple of years old) is a coupon manager that doesn’t rely on any service to collect the coupons.

Thank you for using Stream Notifier throughout the years, giving constructive feedback and recommending it to your friends. I explicitly don’t want to make any replacement recommendations, so if you feel like you need a browser extension that notifies you of streams, use the addons.mozilla.org search and discover possible candidates.

The post Sunsetting Stream Notifier appeared first on Humanoids beLog.

fischertechnik launched the smart home kit last year. A very good move on a conceptual level. Smart home and IoT (internet of things) are rapidly growing technology sectors. The unique placement of the TXT allows it to be a perfect introductory platform to this world.

However, the smart home platform from fischertechnik relies on a central cloud server. This leads to vendor lock-in and challenges to integrate it with other products. Making the cloud your primary interface also means you’re vulnerable to internet outages. Of course the risk really depends on the device type, but you wouldn’t want your thermostat to stop working, just because your internet was down.

fischertechnik is a toy company, though their toy is intended to also serve as a prototyping platform for automation solutions and as education tool. As part of that they, similar to the much better known solution from Lego, have an interface that can run programs to control actors and read out sensors. The latest iteration of this interface is called the fischertechnik TXT. It has an ARM Cortex A8 and can easily be used with custom firmware. This makes it a great platform to build sustainable IoT applications.

Sadly, fischertechnik’s kit doesn’t promote decentralized smart home solutions, instead it opts to provide a proprietary cloud platform:

Beyond that, the TXT Controller can be connected to a cloud server, in which the sensors’ data is saved, and can be retrieved from anywhere in the world. So using a user interface, a so-called “dashboard”, the various sensor data can be permanently monitored, and the movement of the camera can be controlled.

Description of the ROBOTICS TXT Smart Home kit

Mozilla started venturing into IoT recently. They are trying to advocate for better privacy and user freedom by promoting interoperability. These goals are unified in a proposal for a device API that is based on web protocols called “Web of Things”. It is developed in collaboration with smart home and “industry 4.0” manufacturers. This API isn’t necessarily implemented by each device directly, some devices are too low powered to provide a web server or are using other successful local mesh networks like Z-Wave or Zigbee. In these cases a gateway that is connected to the user’s local network would provide the web thing API.

The web thing API breaks physical devices down into three attributes: properties, actions and events. Properties as stateful values that can both be changed by the device and the user (I’ll often refer to the user as the client). A device can however declare a property as read-only or add input value restrictions. Actions let the user execute an action on the device that either isn’t stateful or affects multiple properties. Lastly, events are fired by the device to indicate a momentary effect that is not reflected in the state. Above that is a capabilities system, with which devices can indicate the semantics of their features. For example a light bulb can advertise itself as being a “Lamp”, it’s brightness property will be a “BrightnessProperty” and the power toggle will be an “OnOffProperty”. These allow clients to expose appropriate UIs and behaviors for devices. The protocol is available over HTTP(S) and optionally WebSockets for real-time communication of changes.

Mozilla is developing both a reference gateway to control devices using this protocol and reference implementations in multiple languages to build web things with. The “WebThing gateway” also has an adapter system to bridge other smart home protocols to the web of things data model.

A TXT is easily powerful enough to run the web server required for the web thing API on its own and be a proper web thing. I’ve taken the python APIs from the community firmware and Mozilla’s python web thing library to create an app that can turn the TXT into a web thing. It currently exposes all inputs (I1-I8), outputs (M1-M4/O1-O8), telemetry of the TXT, the camera as well as the counter inputs (C1-C4) as properties. Only the actor properties are writable. Further it provides actions to play a built-in sound and reset a counter input.

The fischertechnik solution for smart home functionality uses MQTT. Why not just adapt that to the web of things? MQTT is only a protocol and the actual packets aren’t standardized. Sadly fischertechnik’s solution is designed to communicate with a cloud solution from a third party. Thus, you’d would have to figure out how to make the TXT connect with a custom MQTT consumer. All in all, I decided it was much simpler and resilient to write an implementation of the web thing protocol that runs on the TXT.

The “WebOfTXT” app, the application connecting the TXT to the web of things I wrote, offers configuration for the sensor and actor types connected to the TXT. After the configuration is set, the user has to start the web thing server. Before starting the server, the schema for the thing is built based on the selections from the user. A QTimer is started to get updated property values. As an exception, the actor properties are only changed by clients of the web thing API, setting the speeds or levels for the outputs. The two actions trigger the respective functions in the TXT API based on the provided inputs. All inputs from API clients are verified by the web thing python package to be valid inputs. For example the actor levels are checked to be integers in the allowed range for the actor type. These limits are also published as a schema in the web thing API, so clients can also expose these limits to the user.

A special case is the camera. It is exposed as an image snapshot. However the image is not served via the web thing API. HTTP can already handle images just fine and there’s no need for scaffolding information. Further an image can’t be written, only read. Thus the image property just points to an image file hosted by the TXT. The TXT updates this image about ten times a second. However refreshing it is entirely up to the client and no notification is sent when a new image is available. For all other properties, the web thing sends an event whenever a property value is changed, so the client can immediately display the updated value.

I found out, that starting the web thing server in the main app thread blocks the TXT UI. Thus the web thing server is started in a separate python thread with its own event loop. Otherwise, the application could not be stopped from the TXT, since it would no longer be registering touch screen inputs.

fischertechnik’s smart home solution is focused on getting the user up and running with little effort and little first party infrastructure. It is relatively simple to build solutions for the TXT that can be used in conjunction with other smart home or IoT devices, though. I believe building your own solution is even more valuable for education on the topic, for example because you can easily consume the data yourself afterward. Bringing up privacy, security and interoperability in IoT education is important, especially considering major manufacturers are avoiding the topics.

Source code of my python bridge can be found in the freaktechnik/WebOfTXT GitHub repository. The python script is only about 750 lines of code, however it requires a couple of python packages to do the heavy lifting. Those are installed by the build.sh script, which produces a ZIP file that can be uploaded to the TXT via the web UI of the community firmware. This should be a good starting point to create a more specific web thing with the TXT and annotate it with the proper capabilities. I want to thank all Mozilla employees and friends from the fischertechnik community that have helped me during the creating of this app, as it was my first time seriously writing python.

A slightly shortened, fischertechnik-specific variant of this article was published in the ft:pedia 1/2019 (pages 64-67)

The post Sustainable smart home with the TXT appeared first on Humanoids beLog.

Developers like to help other developers. Many times via text chats like IRC, Slack, Gitter or Discord.

This article is mainly aimed at developers asking for help. However it can also be applied to other situations, even if the concepts don’t fully apply.

Now for the short overview for the very impatient, though especially those should take the second to last point to heart and read the full article:

• Directly state your question
• Don’t ask for 1:1s
• Answer inquiries in full detail
• Read linked resources thoroughly
• Reference documentation and examples when asking
  

Before I get into the details, I want to mention that I have been helping all kinds of developers via text chat for a couple of years on multiple different platforms. Some days I’m a bit more patient, other days I’ll just give up. Don’t be angry at people when they don’t want to help you anymore, they’re doing this (most likely) for free!

Directly state your question

The solution is not to ask to ask, but just to ask. Someone who is idling on the channel and only every now and then glances what’s going on is unlikely to answer to your “asking to ask” question, but your actual problem description may pique their interest and get them to answer.

Jari Komppa

You may often be tempted to ask if anyone’s available or understands your framework. Don’t (ask), people will help if they can. If you ask to ask first, they may not even get involved, because there’s no problem there that their brain wants to solve.

When stating your problem, try to refrain from explanations like “it doesn’t work”. It not working is obvious to the helper, otherwise you wouldn’t have to seek help. Describe what you’re trying to do and what’s actually happening, just like you’re reporting a bug. Especially when dealing with programming, it often helps to include an excerpt of what you’re currently doing that’s not working. Stack overflow has a good guide explaining how to craft an answerable coding inquiry. Then there’s this guide to help you with sharing code.

Don’t ask for 1:1s

When asking for help in a multi-user chat room, you should also expect to troubleshoot your issue in that chat room.

If you’re insisting on getting help in a 1:1, I can recommend looking on Twitter; some people love to help you in their DMs.

You don’t want to share your code with a public chat room for confidentiality reasons? You probably shouldn’t share it with one random stranger from the internet either. Just saying.

Answer inquiries in full detail

You will get asked follow-up questions, because you probably left out something or there are specific conditions for a bug. Answer these questions in the fullest detail possible. Pay close attention to the wording, experienced helpers will ask well formulated questions.

Read linked resources thoroughly

Before going on and assuming, completely read linked documentation, guides and so forth. Don’t just skim the contents, read thoroughly and try to understand. Just like questions you’d get asked, documentation uses very precise language to describe things.

It’s fine if you don’t understand the documentation. You can ask specific questions about it so that you can learn to eventually read the documentation yourself without having to ask for help. That’s the end goal, after all. You don’t want to end up always having to ask other people to solve a problem. Try to develop new skills when asking questions; especially understanding documentation is a very valuable skill.

Reference documentation and examples correctly

When asking for clarification on details you don’t understand, try to reference to examples or documentation, that way helpers don’t have to try and figure out what exactly you’re talking about.

To the helper it will often be ambiguous what you are talking about without that, since they potentially know much more about the language or library. That’s why they can help you after all. Sometimes they’re just your rubber duck to bounce things off of and spot the missing semicolon (P.S.: please use static code checking like linters).

Tell people when their suggestions are too complex

Developers are usually very excited about new things. New syntax, new libraries, new patterns, they’ll suggest all of them to you. While it’s nice to learn about them, try to understand your problem in the technologies you know first before exploring new grounds as an alternative solution. Sometimes it will not be possible to solve your issue with things that you are familiar with.

Usually this kind of “technology creep” is easy to spot when descriptors like “nicely” or “elegantly” are used to describe a solution. Or when they ask you about the specific environment you’re in. Just to be clear, these aren’t absolute indicators, and you shouldn’t be afraid to learn a little bit of new technology. After all, you want to learn and improve and not just stay stuck with the things you already know.

In short, tell people when they’re pushing too many new things on you that you can’t follow. Usually there’s a simpler solution than to use three new syntax features and two new libraries.

Lastly, please keep this article in mind. Not only to make the lives of helpers easier. One day you will hopefully be helping others, and linking them to such a resource is surely very helpful!

The post Getting Help in Text Chat appeared first on Humanoids beLog.

Emotes are a very important feature of Twitch chat culture. Because of this, there aren’t just emotes you get from subscribing to broadcasters, but also extensions like BetterTTV and FrankerFaceZ that add more emotes to be used in Twitch chat.

Many broadcasters have an image in their panels showing the emotes that are available for their channel. Usually an outdated image, especially since subscriber emotes can be updated much more frequently.

Twitch Extensions are dynamic panels that can update themselves. So sure enough, I decided to solve this by making the Twitch extension “Emotes Showcase” that just displays the channel’s emotes, that simple.

What does it do?

Of course displaying emotes isn’t very simple. Because Twitch emotes are a very big ecosystem. Not only does the panel display subscriber reward emotes if your channel has any, but it also shows any BetterTTV or FrankerFaceZ emotes that are enabled for your channel.

The focus is on your sub emotes when they exist, the BTTV and FFZ sections are collapsed in that case. Viewers can still expand the sections to discover the additional emotes they can use with BTTV and FFZ.

The data used to display emotes is based on the subscription emote sets that twitchemotes.com provide in their API. The emote sets are updated every 24 hours. If you’ve just gotten your first emotes it may take a couple hours for the extension to display them. BTTV and FFZ are directly based on their API.

Why make it now?

I had the idea for this extension since quite some time. And amazingly no one had created it before. A Twitch Extension developer jam, created by LuckyNoS7evin, finally gave me a reason to build it. And someone else also created a similar extension for that jam.

It having such a simple feature set made it perfect for an extension jam with limited time.

Will it ever change?

There are a lot of things I want to add to the extension. Currently it is completely configuration free. Once Twitch lets me store configuration on their systems I’ll add various configurations options to define what to show and how. I’m also planning to add a force refresh button that lets you refresh all currently displayed panels so viewers get the latest sub emotes when a new one available.

I want to use this now, tell me how!

You can get the Emote Showcase extension from your Twitch dashboard. Don’t forget to add it to an extension panel after installing it! As with most of my projects, the source code is available on GitHub.

The post Showcase all your Twitch emotes appeared first on Humanoids beLog.

Bot Land is a browser-based game that is still in development. You can learn more at https://bot.land. I streamed creating this tier list. You can watch the VOD of it on YouTube: https://youtu.be/F-PUJEErrQA

Attack/Offense

Vanilla Lasers

Pros

• Blind fire

Cons

• Countered by reflect
• Not a good secondary weapon

Attacking

Any defense that doesn’t reflect they’re great against.

Defending

Fine, very easily countered though.

Missiles

Pros

• Fire in every direction

Cons

• Countered by reflect
• Don’t do enough damage

Attacking

Ok, not great. Vulnerable to weapons with a bigger range, and vulnerable to weapons with a smaller range.

Defending

Good at being a secondary weapon on Artillery bots, for example.

Artillery

Pros

• Biggest range
• Lots of damage
• They splash damage adjacent tiles
• They don’t care about reflect

Cons

• Travel time
• Countered by teleport, melee

Attacking

Great against defenses that don’t move much.

Defending

Ok choice for defense, but not great against many attack patterns. You will need some other bots that “pin” the attackers, so you can hit them with artillery. Usually a secondary weapon vastly increases the effectiveness of artillery bots. I’d recommend zappers or missiles as secondary weapon.

Zapper

Pros

• Works while you’re using another weapon
• Hits cloaked targets (since it’s untargeted)

Cons

• Has to be used at short range

Attacking

Must have for melee bots, since it’s free extra damage.

Defending

Ok if you like the attacker to be close to you (which you shouldn’t like), and thus not too great because they take a slot that you could use for support hardware.

Melee

Pros

• Deals good damage
• Good counter to most ranged damage if you can get close

Cons

• Bot has to get close to enemies
• Weak against artillery/other ranged damage
• Countered by bigger range (artillery etc)

Attacking

Great against most defenses. Works well with zappers, doesn’t need any other weapons. Gets even better if you move faster (teleport, thrusters)

Defending

Again, ok if you like the attacker to be close to you (see zappers, which are better at that)

Landmines

Pros

• Explode enemies, I guess…

Cons

• Enemies have to step on them
• Needs good scripts

Attacking

Very, very niche. But can be ok.

Defending

Weak, really hard to use effectively.

Inferno Lasers

Pros

• They burn things
• Easier to use than normal lasers
• Keep dealing damage to cloaked enemies/de-cloak enemies

Cons

• They don’t deal as much hit damage
• Countered by reflect

Attacking

Have edge cases where they are better than normal lasers, in general they are about the same though. Maybe a bit easier to use.

Defending

Similar to lasers. Not really better or worse.

Inferno Zappers

Pros

• They still zap
• Do fire damage (de-cloak, enemy can move out of range)

Cons

• Deal less initial damage
• DoT on zapper is less valuable
• Reflect

Attacking

Ok. Great with melee attack bots

Defending

Like normal zappers, not that great. But better against some attack strategies as a close quarters weapon, since you keep dealing damage to enemies that retreat from directly attacking you.

Stunning Lasers

Pros

• Unique effect of stunning.

Cons

• Requires a strategy tailored to them, due to reduced damage
• Countered by reflect

Attacking

Hard to use for attacks, require other damage sources to work.

Defending

Good for ranged defenses (which is really the only good kind of defense), even when they just hit random targets.

Vampiric Lasers

Pros

• Heals your bot (without taking up an extra slot or turn)

Cons

• Deals less damage per hit
• Countered by reflect

Attacking

Less wasted turns when attacking chips. Ok, drop-in replacement of lasers that can be better in edge cases.

Defending

Better healing sources for defense than sacrificing laser damage.

Acceleration Missiles

Note: haven’t used the new, revised acceleration missiles

Pros

• Can eventually deal a lot of damage.
• Sound really neat on paper

Cons

• Not great in practice
• Countered by reflect

I don’t think you should be using them. Neither for attack nor defense.

Support/Defense

Cloaking

Pros

• Enemies can’t see you when you’re cloaked (while you don’t walk into them)

Cons

• Doesn’t work well with melee or Zappers
• Cooldown – can only cloak every now and then

Attacking

Ok for attacks, requires very specific usage, don’t use with melee!

Defending

Not as valuable, so very niche.

Regen

Pros

• Restores life

Cons

• Only restores a little life

Attacking

Don’t bother, if you’d survive due to regen, another form of support may have been more effective.

Defending

Don’t bother either, use the chip.

Teleport

Pros

• It moves your bot far quickly.

Cons

• Has a cooldown

Attacking

Great for retreating, not as good for quick melee or similar.

Defending

Great for getting in the first hit, retreating is less of an issue/consideration.

Thrusters

Pros

• Makes you attack faster
• Can potentially kill enemy just before it would kill you (1v1 duel winner)

Cons

• Not as good if your enemy has more thrusters (or haste)
• Not as useful if you have teleport

Attacking

Good if you have spare slots

Defending

Good if there’s a thruster meta, but really only use it for spare slots, haste chip has similar effect for free.

Reflect

Pros

• Reflects ALL the things

Cons

• Sometimes on cooldown
• Never a guaranteed reflect

Attacking

Use whenever you encounter things that you can reflect. Prefer over any other defensive hardware. I have two versions of many bots, one with and one without reflect, since reflect is useless when attacking any defense without lasers or missiles. What to replace reflect with depends on the strategy for the bot.

Defending

Reflecting wall in front is pretty good, if your bots die quick, all bots should probably have some reflect. If you use reflect, you’ll see more artillery and melee attacks.
Makes it really easy to deal with any strat that uses weapons that can be reflected and you can focus on other strategies.

Shield

Pros

• Shields your bot from damage
• Shield other entities

Cons

• Have to be activated

Attacking

Solid, better than repair for simple strats.

Defending

Depends, can be strong, but active healing by specialized healers is stronger, since those don’t take up a slot or turn.

Armor

Pros

• Reduces damage taken
• Passive

Cons

• Reflect is the better slot filler
• Worse than regen, only absorbs so much damage, while regen can regenerate infinite damage taken (in theory)

Attacking

Not sure why you’re not using reflect…

Defending

At least have the decency to use regen.

EMP

Pros

• Disables weapons

Cons

• Not great against activatables
• Hard to time right, due to range

Attacking

If you’re using EMP as an attacker, I’m sure you have a good reason for it.

Defending

Good to shut down that one annoying thing you forgot to deal with in your defense. Requires some custom code to be effective.

Repair

Pros

• Repair yourself
• Repair others
• Repair CPU

Cons

• Not an AoE

Attacking

Do you want to live forever? If yes, build a bot that retreats and heals (and hope no-one follows you, as in TP away so they don’t see you anymore)

Defending

Use it when you don’t have area repair yet. Use it when you have some CPU cult worshiping your CPU. Have a dedicated bot, probably.

Area Repair

Pros

• Heals things
• AoE

Cons

• AoE (for attacking)
• Doesn’t heal things that aren’t bots

Attacking

Bad, because AoEs when attacking are hard.

Defending

Great. Use it (on a dedicated bot, or just anywhere)

Chips

Nice-to-have, don’t rely on them, generally not game-changing.

The post Definitive Bot Land Beta tier list appeared first on Humanoids beLog.

Most browser extensions do a thing. And they do that thing in their isolated little world. Many of them do their thing pretty well. Many of them are built to do many things. Many of them are built to do just one little thing. But only few of them talk to other extensions to do things.

And I’d like to see more extensions talk to each other to do things.

A Couple of Examples

Running a Task

One of my extensions lets you know when creators are streaming. Some people like using external tools to watch those streams (for many reasons). While I do not want to support a native application that interacts with Firefox to launch the tool, I have instead found an extension that already provides integration with the tool and Firefox and asked nicely to add an API for other extensions to be able to launch the tool. And now I can offer an option to open streams with the tool when that extension is installed. Since this is a feature only few users would be using it is a much more efficient approach for me and it makes it simpler for users of that other extension, since they don’t have to install two native applications to interact with the tool.

There are many other things that other extensions may already be doing and you don’t necessarily want to repeat in your own extension. I can think of interacting with some APIs to share content on a service or applying certain modifications to a page as generic examples.

An API for a Custom Feature

Another one of my extensions plays a sound when a notification is shown. That sound can be customized. It’s something Firefox doesn’t do out of the box. If other extensions also want to have that sound played when they show notifications, and thus make it way simpler for users to get a unified notification experience from Firefox, they can just tell that extension that they are showing a notification.

I initially opted to build a separate extension to play a sound for new notifications because I did not want to add my own solution in every notifier extension I maintain. And because some operating systems already play a sound, so why should I always just have that feature built into my extension, when it’s easily modularizable and users can opt into it by installing an extension.

There are many other extensions, especially in the tab management category that do very custom stuff with how they handle native Firefox features. Giving other extensions a way to directly interact with those additional features opens new and exciting opportunities.

Adding more Functionality

The popular Twitch modification extension FrankerFaceZ has an add-on pack for more emotes and additional customizations. The add-on pack fully integrates into the UI of the base extension.

Other classic examples for this include Firebug extensions (which are now just normal developer tools extensions) and extensions adding features to ad blockers.

How can Extensions Integrate with Each Other?

There is a well hidden feature in the very basic messaging API for WebExtensions: you can send a message to another extension. And of course that extension can then reply to your message, or send you more messages, since it gets your extension’s ID as part of the sender info.

The API in question is the sendMessage API in the runtime namespace. You just specify the ID of the extension to send your message to as the first parameter. Extensions can also open ports between them to exchange more than just single messages in a context. To listen for incoming messages from other extensions a separate event listener onMessageExternal is required. The same goes for incoming port connections with onConnectExternal.

When listening you do not only get the message from the other extension, but also which extension is interacting with you. This allows you to allow extension-based filtering to reduce the threat of abuse or for users to further customize their experience.

The actual messaging protocol has to be defined by the individual extensions. The protocols I’ve used so far are both documented in the README of the extension.

Some extensions expose an API in the page they modify with a global object. This lets users customize extension behavior via user scripts or other extensions. I don’t have an example of any extension that does this and has a documented API.

I’ve created awesome-extension-apis as a curated resource for extensions that have a messaging API for other extensions to use. Please add your extensions with messaging APIs to that list!

Why Should they Integrate with Each Other?

Primarily to reduce configuration repetition for users. At least that’s what I think integrations are primarily useful for. Instead of every extension providing its own solution for a problem they can reduce it to a single point of configuration for the user by all requiring the same extension.

This is not really fitting for core features, since users have to install a second extension. But to expand the abilities of your extension I think it is fine to ask users to install a different extension. Especially for less extension related things like a notification sound. It may be a harder sell for integrations with services, like Streamlink, however I feel the problem is similar to requiring a native application. It may even be a little less severe, as installing an extension requires a lower trust level from the user and is easier.

The post Browser Extensions should Work Together appeared first on Humanoids beLog.

In early 2016 the Swiss Mozilla Community met up to discuss goals and community structure. During a discussion on the state of our Twitter account @MozillaCH I pitched the idea of having a tool to manage content like replies with contributors instead of just people with direct access to the account.

Finding a solution

I then took the task of investigating existing solutions. After not finding anything that really fit my vision of a system to manage tweets going through an editorial flow, I decided to build the tool myself. Hopefully with contributions from others, but given our community size I had no expectations.

I first wanted to automate adding Tweet drafts and then sending them with a Trello board. Just then GitHub released project boards. We already use GitHub for community organization, so managing tweets wasn’t unthinkable.

Moving to GitHub also has the advantage of reducing the barrier of entry for potential tweet writers. All you need is a GitHub account. This expanded the vision of the tool allow contributors to suggest content for the Twitter account.

How it works

The basic idea is that we have a card per planned tweet on a project board. We move that card through multiple stages on that board, until it is in a column for content that should be tweeted. Other columns can be sources for tweets that automatically generate a card when new content is available, like an ICAL feed from REMO or new Twitter mentions. Other columns may be for editorial management, like proof reading.

Every card has an issue attached to it, which has information like the tweet content, when it will be tweeted or what tweet it is a reply to. For the tool to understand the issue, it needs to follow a standardized format. To make interaction with the tool easier, it comments when content is not valid. Comments also serve as reminder for unfinished content that is scheduled to be tweeted soon.

Most of the content sources and management functionality is individually controllable, including if the tool should use them at all.

The tool is Node application that interacts with issues, project board and the twitter account. Configuration is a static JSON structure, or via the environment, since it can also be deployed via a Docker container.

Starting to use it

It took about a year until the tool was ready enough. And then breaking it a few times and fixing it before we finally deployed it for use with our Twitter account. And even after that we’ve found some bugs and fixed some. In short, the tool is still in its infancy.

We use a buffer-like system, where only one tweet is sent out every day for unscheduled content. We may need to tweet more than once a day. For the moment it seems our content gets spread out nicely thanks to this system, instead of compulsive tweets & retweets in the middle of the night.

You can now contribute content for the @MozillaCH Twitter account in the mozillach/tweets repo on GitHub. We have tweet curators that will then help you create a valid tweet and move it through the content queue.

The future

Meanwhile a test is running for the @MozillaReps Twitter account, and hopefully others will follow.

My goal is to finish test coverage for the tool, to improve its reliability. Eventually I want to add a web interface to make management of settings easier. Others have suggested making it possible to post to other services like Facebook or Telegram Channels.

Sadly the documentation is pretty basic currently, there is a README explaining the configuration, but actual functionality is only documented in our CONTRIBUTING.md for the @MozillaCH tweet board.

Source code is of course available, in this case on GitHub: https://github.com/mozillach/gh-projects-content-queue/

Feel free to install it for your own Twitter accounts, file issues and pull requests. I believe this is a very accessible, new way to allow collaboration on Twitter content.

The post Democratizing our Twitter Account appeared first on Humanoids beLog.

It’s been quite a while since I last posted on here, and interestingly about the same topic: code coverage analysis in Firefox extensions. And since then Firefox has gotten a completely new extension system. I’ve been really busy porting my extensions and not writing blog posts.

The add-on SDK conveniently came with a test runner built into the JPM command line tool. WebExtensions don’t, which forces the choice of test runner on the developer.

If you use a traditional test runner, you’re probably not really struggling with code coverage. You may have had to do a little bit of research to properly mock the environment of the test runner, so your extension code can run in it, most likely ending up with something that exports the DOM globals from JSDOM and some stub of the WebExtensions API (my choice is sinon-chrome). And then you’re done.

const browserEnv = require("browser-env");
const browser = require("sinon-chrome/webextensions");

browserEnv();
global.browser = browser;

Trouble is, I like the fancy modern test runners, which run tests in parallel. Of course we could use the global injection approach there. However we would sacrifice running tests in parallel inside a single test file. While we may still be able to run multiple test files at once depending on the test runner (my choice is ava).
So how can we run them in parallel then? The extension files all need a window global and a browser/chrome global. Sure, but luckily JSDOM by default doesn’t export the window global to the node global, instead it creates a sandbox. And into that sandbox we also inject the WebExtension API stubs and whatever DOM APIs we need that JSDOM doesn’t come with. Finally we execute the file to test in that sandbox (using eval) and pass in the window as context to the test.
const fs = require("mz/fs");
const { JSDOM } = require("jsdom");

/**
 * Creates sandbox with DOM and WebExtensions APIs and loads instrumented files in it.
 *
 * @param {[string]} files - Paths of files to load in the DOM sandbox.
 * @param {string} [html="<html>"] - Document to create the DOM sandbox around.
 * @returns {object} JSDOM global the files are loaded in.
 */
const getEnv = async (files, html = "<html>") => {
 const dom = new JSDOM(html, {
   runScripts: 'outside-only',
   virtualConsole
 });
 dom.window.browser = require("sinon-chrome/webextensions");
 // Purge that instance of the browser stubs, so tests have their own env.
 delete require.cache[path.join(__dirname, '../node_modules/sinon-chrome/webextensions/index.js')];
 for(const file of files) {
   dom.window.eval(await fs.readFile(path.join(__dirname, file), 'utf8'));
 }
 return dom;
};

Sounds simple enough, and results in us getting a sandbox per test. A lot of overhead, just to run tests in parallel. We haven’t even started with overhead though.
Like in the article for the add-on SDK, to get coverage, files have to be instrumented and coverage has be read from them. This setup isn’t doing that yet. Instead it currently loads the tested files using node’s fs module (actually, using a promisified version) and then evaluates them inside the sandbox. At no point could my coverage tool of choice, nyc, instrument them or read coverage.
To sum up, we have to manually instrument the files when loading them using nyc and then manually export the coverage results so nyc can generate a report. Now, the instrumenting bit is actually an API nyc offers. Trouble is the reading the coverage bit.
Let’s start with the simple task, instrumenting. The following code snippet replaces the readFile call in the getEnv function from before. It only instruments files using nyc when nyc is being used to run the tests, else it just reads the file.
const fs = require("fs/mz");
const cp = require("child_process");
const util = require("util");
const ef = util.promisify(cp.execFile);

/**
 * Returns the source of the specified file, instrumented when running nyc.
 *
 * @param {string} sourcePath - Path to the file that should be loaded.
 * @returns {string} Potentially instrumented source code of the given file.
 */
const instrument = async (sourcePath) => {
 if(!instrumentCache.has(sourcePath)) {
   if(!process.env.NYC_CONFIG) {
     const source = await fs.readFile(sourcePath, 'utf8');
     instrumentCache.set(sourcePath, source);
   }
   else {
     const instrumented = await ef(process.execPath, [
       './node_modules/.bin/nyc',
       'instrument',
       sourcePath
     ], {
       cwd: process.cwd(),
       env: process.env
     });
     instrumentCache.set(sourcePath, instrumented.stdout.toString('utf-8'));
   }
 }
 return instrumentCache.get(sourcePath);
};

Sadly we can’t just share the coverage global with the JSDOM scopes. Instead we have to manually collect the coverage and hand it over to nyc.
Luckily nyc reads all “.json” files in the “.nyc_output” directory as coverage results. Thus we can just write the coverage results to a uniquely named JSON file in that directory once tests are done (oh, and we close the window to clean up all garbage):
const fs = require("mz/fs");
const mkdirp = require("mkdirp");
const util = require("util");
const mk = util.promisify(mkdirp);

/**
 * Saves coverage to disk for nyc to collect and cleans up the JSDOM sandbox.
 *
 * @param {JSDOMWindow} window - Window global of the sandbox. Window property
 *                               on the global getEnv returns.
 * @returns {undefined}
 */
const cleanUp = async (window) => {
 if(process.env.NYC_CONFIG) {
   const nycConfig = JSON.parse(process.env.NYC_CONFIG);
   await mk(nycConfig.tempDirectory);
   await fs.writeFile(path.join(nycConfig.tempDirectory, `${Date.now()}_${process.pid}_${++id}.json`), JSON.stringify(window.__coverage__), 'utf8');
 }
 window.close();
};

Update (2018-11-10): in nyc 13.1.0 tempDirectory was changed to tempDir.
Code in this article are excerpts from the test suites I use for two of my extensions. Advanced GitHub Notifier has the fancy one we built throughout this article, which essentially creates a sandbox per test and Live Stream Notifier uses a global scope-polluting setup. Both setups have their drawbacks. Polluting the global environment means that tests depending on the state of the API stubs or similar have to run in series. However it is much faster than initializing a whole DOM sandbox for every test and requires a lot more setup.
The post Code coverage reports for WebExtensions appeared first on Humanoids beLog.
]]>
					
		
		
		3114	
		
		
		https://humanoids.be/log/2015/11/checking-code-coverage-of-add-on-sdk-extensions/
		
		
		Wed, 04 Nov 2015 22:18:08 +0000
				
		
		
		
		
		
		
		
		
		
		http://humanoids.be/log/?p=3057

					
Code coverage statistics are very useful. They tell you how much of your code never gets executed during the unit tests. So you always know what to write unit tests for, because 100% coverage doesn’t exist. And sometimes you can’t cover all code. That’s why normally you are happy, if the coverage is above a … 
Continue reading "Checking Code Coverage of Add-on SDK Extensions"
The post Checking Code Coverage of Add-on SDK Extensions appeared first on Humanoids beLog.
]]>
										
Code coverage statistics are very useful. They tell you how much of your code never gets executed during the unit tests. So you always know what to write unit tests for, because 100% coverage doesn’t exist. And sometimes you can’t cover all code. That’s why normally you are happy, if the coverage is above a certain thresh hold, but having a higher coverage is always ok
One of the typical coverage tools for CommonJS based code is istanbul. Sadly it doesn’t work just out of the box, because the coverage calculations and tests aren’t run in the same JS engine. The tests run in Firefox and the calculations happen outside in NodeJS.
By default Istanbul uses a global variable to store the coverage information. The problem begins with the CommonJS environment of the Add-on SDK not having a global environment. To work around that, I’ve made a module that exports an object which istanbul then treats as its global object. But to do that, the Instrumenter, the thing that makes your code inspectable during unit tests, has to be modified to use that module instead of the Node global. Luckily this is just a modification of one line in a function in the prototype of the Instrumenter. This happens in the main module of the istanbul-jpm package, which I created.
/* The second item in the array differs from the normal
 * Instrumenter, everything else is the same. */
code = [
    "%STRICT%",
    "var %VAR% = require('istanbul-jpm/global').global;",
    "if (!%VAR%.%GLOBAL%) { %VAR%.%GLOBAL% = {}; }",
    "%VAR% = %VAR%.%GLOBAL%;",
    "if (!(%VAR%['%FILE%'])) {",
    "   %VAR%['%FILE%'] = %OBJECT%;",
    "}",
    "%VAR% = %VAR%['%FILE%'];"
]

Having a custom module providing the “global” scope also makes bridging the gap between Firefox and Node really easy. It just serializes the “global” object when the extension is unloaded (after all unit tests are done) and writes it to disk. Funnily enough the module is called global. There is a module for node that then reads and parses that file called global-node.
/* Loading the collected coverage data into the node
 * environment. */
global.__coverage__ = require("istanbul-jpm/global-node").global.__coverage__;

Only the global.js from the istanbul-jpm package is needed during unit tests, everything else can safely be .jpmignored. Sadly you have to do that yourself, since JPM doesn’t respect the .jpmignore files from extensions.
And that’s all the extra magic needed. The rest is magic that needs to happen whenever you collect coverage data. But because generating coverage statistics is quite complicated I’ll explain the other steps too:
Before the unit tests are ran, the Instrumenter adds the tracking logic to your modules (normally everything in your lib folder and maybe in your root). I haven’t bothered to figure out a way to do coverage analysis for content scripts, by the way.
But for coverage data to be collected the modules generated by the Instrumenter have to be loaded in the unit tests and not the normal modules. Common practice is to create a require-helper module, that checks an environment variable and either loads the module from its actual location or redirects it to the instrumented version.
An example implementation for instrumented modules being in the /coverage/instrument/lib/ folder, the following code does exactly what is described in the previous paragraph. If the environment variable JPM_MEASURING_COVERAGE is set, it redirects to the instrumented versions.
const system = require("sdk/system");

module.exports = function (path) {
    if(system.env.JPM_MEASURING_COVERAGE)
        return require('../coverage/instrument/test/' + path);
    else
        return require(path);
};

After the unit tests a report has to be generated. This can have many forms, and really depends on what you want to do with coverage statistics.
I use grunt to manage all of this, a bootsrapped Gruntfile.js can be found in the README,md of the istanbul-jpm module. There is also the monstrosity, that is the Gruntfile.js of my jtvn extension.
The istanbul-jpm module helps close the gap between the normal Node environment and JPM tests for coverage analysis with istanbul. It’s pretty young, but there’s not much that can go wrong. Give it a try!
The post Checking Code Coverage of Add-on SDK Extensions appeared first on Humanoids beLog.
]]>
					
		
		
		3057	
		
		
		https://humanoids.be/log/2015/08/testing-add-on-sdk-extensions-on-travis-ci/
		
		
		Mon, 17 Aug 2015 09:51:32 +0000
				
		
		
		
		
		
		
		
		
		
		
		
		
		http://humanoids.be/log/?p=3014

					
Recently Mozilla restricted access to ftp.mozilla.org, instead you can download nightlies from archive.mozilla.org. Sadly this broke existing methods to download Firefox nightly for automated unit tests on Travis CI. I’ve written a node module, that takes care of downloading nightly versions of Firefox desktop and Android. .travis.yml for Desktop sudo: false language: node_js node_js: stable … 
Continue reading "Testing Add-on SDK Extensions on Travis CI"
The post Testing Add-on SDK Extensions on Travis CI appeared first on Humanoids beLog.
]]>
										
Recently Mozilla restricted access to ftp.mozilla.org, instead you can download nightlies from archive.mozilla.org. Sadly this broke existing methods to download Firefox nightly for automated unit tests on Travis CI. I’ve written a node module, that takes care of downloading nightly versions of Firefox desktop and Android.
.travis.yml for Desktop
sudo: false
language: node_js
node_js: stable
env:
  global:
    - DISPLAY=:99.0
    - JPM_FIREFOX_BINARY=$TRAVIS_BUILD_DIR/../firefox/firefox
before_install:
  - sh -e /etc/init.d/xvfb start
  - npm i -g get-firefox
  - get-firefox -ecb unbranded-release -t ../
before_script:
  - npm install -g jpm
script:
  - jpm test
Update: I changed the export commands to setting the variables with the .travis.yml env property, because it feels cleaner.
Update 2: Switched to the Firefox from travis.
Update 3: With unsigned builds being a thing and travis not supporting them yet, I’ve switched back to using get-firefox for desktop tests too.
.travis.yml for Android
Since jpm-mobile is currently broken, this clones a fixed version and installs it. Further, timing for the Android Emulator is somewhat important, so you might have to fiddle with the order and content of the before sections.
sudo: false
language: android
android:
  components:
    - platform-tools
    - tools
    - android-19
    - sys-img-armeabi-v7a-android-19
env:
  global:
    - DISPLAY=99.0
    - JPM_FIREFOX_BINARY=fennec
    - JPM_ADB_PATH=/usr/local/android-sdk/platform-tools/adb
    - TMP_DIR=/tmp
before_install:
  - sh -e /etc/init.d/xvfb start
  - mkdir -p $TMP_DIR/sdcard
  - mksdcard -l jetpackSdCard 1024M $TMP_DIR/sdcard/jpmsdcard.img
  - rm -f $TMP_DIR/sdcard/jpmsdcard.img.lock
  - echo no | android create avd --force -n jpm -t android-19 --abi armeabi-v7a
  - emulator -avd jpm -sdcard $TMP_DIR/sdcard/jpmsdcard.img -no-audio -gpu off -no-boot-anim -noskin &
  - android-wait-for-emulator
before_script:
  - npm install -g get-firefox
  - git clone --depth 1 -b fixes https://github.com/ncalexan/jpm-mobile.git $TMP_DIR/jpm-mobile
  - cd $TMP_DIR/jpm-mobile
  - npm link
  - cd $TRAVIS_BUILD_DIR
  - get-firefox -c -p android -t $TMP_DIR/fennec.apk
  - adb wait-for-device
  - adb emu input keyevent 82 &
  - adb install $TMP_DIR/fennec.apk
script:
  - jpm-mobile test -v --adb $JPM_ADB_PATH -b $JPM_FIREFOX_BINARY
The post Testing Add-on SDK Extensions on Travis CI appeared first on Humanoids beLog.
]]>
					
		
		
		3014