The Priority Pyramid is a tool I use to stay on track with new consulting clients. It prioritizes how, who, and what I engage in at any given time. It can be overwhelming when thrust into a challenging situation, a code base in dire straits, and a frustrated team. You need a strong pillar of guidance.

This article goes over what parts make up the Priority Pyramid from a high level. I’ll talk about what milestones make up each section and how you navigate back and forward between the priorities.

When done, you should know how to engage your client’s team and tackle working on a large code base at a frustrated client site with 99 problems.

Why?

Over the years, I’ve prioritized what I need to be doing when working my consulting clients. There are countless things I need beyond “coding”, and all have to do with “people”. Code in its own right is challenging enough. Add people to the mix and it gets more complicated than it already is.

Every engagement is different, even with repeat clients. This is why I like consulting. Meet new people, travel new places, learn new things.

While I enjoy and thrive on improv, what I didn’t like was not having a set of guidelines to deal with each new client. The business blogs & books I’ve read are pretty dry, and don’t really focus on the people issues coders encounter and have to deal with. Most also assume YOU are the one creating & maintaing the app, not fixing/modifying a large one extremely behind schedule. It’s either sales, management, or product management. This includes the consulting books. Some approach it from a larger consulting perspective, where you come in, listen to the client’s problems, write up a plan, and pass it off to India. If you are involved, it’s just to either head off major disaster and/or provide face time for the client.

This is NOT what I, and my colleagues I’ve worked with over the years do. We’re in the trenches, working alongside our clients, playing the politics games whilst coding.

So, I made my own rules for engagement, prioritized based on importance, with verification points to ensure I’m on the right track. If I cannot work towards getting through all tiers of the pyramid, I will not be working for the client long, whether by my choice or theirs. It’s important I flush out these red flags early and tackle them full bore.

That… and it has D&D connotations.

Quantitative vs. Qualitative Foundations

I’ve used both quantitative and qualitative results from case studies, white papers, and my own experience to base this set of priorities on.

Mythical Man Month/Brooks’ Law: “Adding manpower to a late software project makes it later”. Some of the bad projects I’ve been on had too many people on them. Your goal at that point is to delegate refactoring to the team to compensate, lead with good architecture, and leverage your trust currency to guide the various teams into a direction you need them to go. Also known as software chaos management. This requires you be at a high enough level to be in charge of various teams. If you’re not, God help you.

Code Review: One person, reading code for up to but not exceeding 1 hour finds more code defects than unit testing. This is not peer review, but that is also effective. Citations: Code Review at Cisco Systems, Klocwork/Forrester “The Value and Importance of Code Reviews”, and some other stats from Code Complete. Keep in mind while most of the studies mention that defects do not significantly decrease after 1 person, the ability to learn from your peers and getting architecture suggestions are very well documented in the blogosphere.

Cheaper to Find Defects Early: Whether you are ruthless about upfront requirements gathering and/or initial architecture, or are adamant about short development iterations that includes scope reduction, both allow the ability to find more defects early. It’s easier to both find, and fix problems earlier than later for a variety of reasons (code size, feature acceptance, contractual obligations, etc). Beyond the IBM studies back in the 70′s here’s an excerpt from High Quality Low Cost Software Inspections. Also, crazy charts from Capers Jones (they have some other good studies). This is why I’m so adamant about getting the visual design (wireframes/design comps/ux/information architecture, etc.) right from the get go, even if 6 months into the project. Lowest hanging fruit and saves tons of time/money.

Distributed Development (PDF): The result that physical distance between developers doesn’t matter. Different room/building/country… not a big deal at all. What matters is how far the software developer(s) are from the stakeholder(s) in the organization management hierarchy. This is extremely important. I am firm believer in both telecommuting as well as on-site visits with clients for communication & leadership reasons. This is central to identifying the power structure of the organization in Tier #2 and garnering trust with those who can help you talk to who you need to talk to in Tier #3. I’ve been ineffective at more than one consulting engagement because I wasn’t  high enough to affect the required positive change.

Toyota Production System: Their continuous improvement philosophy and respect for people are what really ring true to me. While It Takes a Village to build great software, sometimes you have a crowded, drama filled city that you have to work within. Doing your best to get the team motivated to constantly improve, and SEE the improvement occur over time for motivation reinforcement, is just a small way to continually improve. You can only do this if you do in fact respect the people you work with. Empowering leaders amongst those who actually do the work can lead to effective production. You can use these Toyota Process strategies in your own team delegation.

The Priority Pyramid

The 8 steps are in order of importance:

1. Report
2. Understanding
3. Trust
4. Lead
5. Build
6. Explosions
7. Diagnostics
8. Architecture

While it implies 4 are about coding, make no mistake: The Priority Pyramid is all about people. It’s built on your relationships with them and those relationships ensure you can do your job.

Lets break down the tiers the following way: What milestones do you need to complete, how to do you move on to the next tier, and when should you move back? I make the assumption if you’re on a team, you’ll divide tasks appropriately.

A Note on Tier Foundations

Each tier works on top of the foundation of the former. If the foundation is weak, it won’t strongly support the one on top. If you don’t have the trust of management and your co-workers, it’s difficult to lead them. If you don’t have a good understanding of the company and people who work within it, it becomes very difficult to affect positive visual design changes to make architecture easier. It’s ok to be in Tier #8 and realize you need more understanding of how the application will be deployed from Tier #2; just like refactoring code over time, you can continue to strengthen the foundation tiers over time as well.

Tier #1: Report

Milestones Before Arriving

1. You’ve talked to stakeholder(s) on the project to have an idea of what you’re getting into.
2. Verify you actually want to get into it. Some consulting gigs are hard to get out of. Some can reflect poorly on your firm/you even if you do a good job. Make sure you’re willing to make the plunge. We’re looking for red flags here to save you tons of money and stress. I’m not saying be choosy, I’m saying be careful.
3. Where you’re going, who you’re supposed to see, and when.

Milestones

You/your firm has scored a client, you’ve booked your airfare + hotel, and you’re 1st day on-site has arrived. You look sharp, are excited, and looking forward to making a good first impression.

1. First question out of your mouth after small talk should be “Who’s do I report to?”. Yes, it’s ok to follow up this question once you’ve scored more rapport with “Great, so… who’s my boss?”. This ensures you don’t do good things that get you in bad trouble.
2. Second question: “What will make you happy when I leave?”. This helps verify why you’re really here, and what they’re really paying you for. Sometimes this takes alcohol to get the true answer. Sometimes just a neutral setting outside of the office/building.

Successful Exit

Know who you work for and your true goal in life for the next 8 months.

Tier #2: Understanding

The best communicators are good listeners. Your task now is to look people in the eye, lean forward, and listen well.

Milestones

1. Listen: Get people talking, asking questions, and listen to what they say. This is the most important step in the entire pyramid.
2. Goals: You need to understand what the goals are for the project, both stated and real. What are you and the client really trying to accomplish? Answering & documenting this will guide your actions for the rest of the project, overridden only by “Who do you report to?”.
3. Problems: What are the problems and challenges the company has run into? Do any relate to why you’re here? Do they have a thorough list or are there more? Solving problems earns trust, and helps in the next tier. It also validates your paycheck. Why does the client have those problems? Where do you fit in? Who’s accountable for identifying solution(s) for and solving those problem(s)? These are your first challenges in the game.
4. Dissect People: This is also an opportunity for you to start learning who the players are in the game. You need to understand what makes someone tick to best interact with them. Do they like business first, small later or the opposite? Do they have ulterior motives? Can they be trusted? Can they give you valuable intel on the project others won’t? What will it take to make them comfortable enough to tell you more? How can you possibly help them? You’re going to be spending the next 3 to 18 months with these people. Get to know them, make a good first impression, and understand them. These are the people you want on your team, who you want to do certain things for you, who you want to trust you. Once you understand them, you can effectively engage them. Sometimes this is done simply by watching how they interact with others. Ask questions to get them talking; their personality whilst reveal itself. Finally, figure out who reports to whom for real? What is the power structure? You’ll need to play within this framework; best to know how it works for real vs. what the label says on their door.
5. Explore: Finally, code stuff. Time for initial reconnaissance. Taken what you’ve learned, it’s time to verify what you heard is recent, relevant, and can be corroborated with the code. This should still be framed from a “learning the company” perspective. You need to:
1. Learn the Data Model: The VO’s, the Services it calls, and what those business objects actually mean to the organization.
2. Learn The Framework: How is the code architected; what framework did they use?
3. Understand The Story: With another developer, preferably one who worked on the code base and/or your client, ask the developer questions to learn the story. Even the worst code base has a reason how it got there. Like an archeologist, you want to understand this history, the pivotal decisions, or lack thereof, that made it the way it is. You’ll learn the to respect, and pity, even the worst code base once you’re armed with this knowledge.
4. Look For Red Flags: Both the currently benign, and the “holy eff we’re eff’d”. Whether that’s everything being a hashmap in Java, Angular & Backbone both being used in the same code base, or ActionScript 2. You don’t want surprises, you want to frame your perspective of the code base with your client’s. This is a VERY important step. Don’t say it, but talk about it internally in my mind. Find out why there is a disparity in perception.
5. Look For Mines: Like red flags, these are things that down the line of a project, based on what you learned above could go boom. They aren’t a priority but need to be documented.
6. Look For Validation: Did everything you learned in Steps 1 – 4 get validated or where you completely off? If so, continue to dig, and redo steps 1 – 4 to re-verify.

Successful Exit

When you firmly understand the client’s reality, the reality the code reflects, and the real reality once you’ve resolved the two, you’re ready to move on. If they still don’t mesh, keep asking questions & researching.

Tier #3: Trust

Assuming you’ve made good first impressions with those you’ve met, it’s now time to validate those initial impressions in those you’ve met. It’s time to start building trust.

Milestones

1. Provide Immediate Value: Check in code into the company’s source control your 1st week there. Preferably the code fixes a bug. Do NOT try to save the world in the code base. Pick some litter instead. If you clean the entire thing in a day, great, but get your feet wet and see what happens. A lot of drama and things you couldn’t predict happen AFTER you’ve checked in code and it goes into production. Start small and work your way up. Just ensure it’s immediate.
2. Make Professional Friends: Respect those you work with and get to know them. Whether this is small talk that’s sincere or genuine interest, whatever works to build a positive relationship. People trust those they like. Trust is a currency you’ll sometimes have to spend in buckets to make positive headway so start investing in earning some now.
3. Visible Logger: Invest in a visible logger. Even if the code doesn’t currently have problems, when it does, you want it to talk to you. Not only you, but the company. You want to corroborate this communication with your boss. When production explodes, and you know within seconds because your little secret keyboard shortcut debug window shows the error, that inspires confidence. It also provides a kind of transparency to your client when others start using it proactively to find problems. You just empowered a company to work together; well done, that’ll earn some trust. Make the code talk to you so you can trust what it’s saying, then expose this communication channel to your client.
4. Provide Transparency: As you start getting your feet wet, whether this is struggling to get your environment setup so you can compile, or are deep in the bowels of fixing your first bug, let your boss know. Keep them appraised of what you’re doing on regular intervals, even if she doesn’t ask. Come across with confidence; you KNOW what you need to be working on, and you’re informing your boss as such. Additionally, she has a chance to re-prioritize if you need to be working on something more pressing. No communication, and neither of these things happen. If they know and have confidence in you doing what you say you’re going to do, they can play defense and keep their boss off your, and your team’s back.

Successful Exit

Do you think people trust you? Have you checked in working code that fixes a problem and your boss has verified this? Can you even compile? Do you know the names of those you’re working with/near? If yes to all, you’re ready to move on.

NOTE: If you ever lose trust, fall back to continually adding immediate value. Trust isn’t given, it’s earned. If you lose it, the only way to get it back to is re-earn it from scratch.

Tier #4: Lead

At this point you know enough about the goals of the project to start formulating plans on moving forward, whether this is how to implement new features or how to fix/refactor existing ones. You’ve provided enough value to earn a modicum of trust. It’s time to lead people forward.

A manager puts people in a line, puts the guy with the machete at the front, and leads people through the jungle effectively. A leader climbs the tree, and tells everyone they need to go left instead.

Milestones

1. Short Term Goals: You’ve assessed the situation and identified priorities that need to be refactored (using my previous article). You need the team to agree on a common logging interface, both in code and visually. Finally, you need the team to agree on an architecture.
2. Long Term Goals: You’ve identified architecture holes that need to be plugged. Sometimes there are design problems that if changed could save months of work and hardship. You need to start work and documentation on it now. Solidify the long term goal into something defendable, and get the teams input. Be careful how you go about talking about it. While you need client feedback (remember transparency), it must be attainable to company-relevant, short term steps. You can spend trust currency for longer term refactoring/re-architecture endeavors, but I recommend against it.
3. Resources: Sometimes you need to hire additional people, whether from internal client teams, additional consultants/contractors from your firm, or hiring on behalf of your client. This could be staff augmentation or hiring a specific skill set you need/want. Identify these early and ask your boss.
4. Divide, Delegate, and Conquer: Whether the client’s employee’s or your firms, you need to divide up the massive amount of work amongst people. Offer up tasks first, and then assign to those you deem appropriate if no volunteers materialize. Building upon Tier 2 and 3, you should have a gut feeling for who’s best for what task. You cannot save the world by yourself. It may seem that you spend more time aiming people in the same direction then actually walking that direction yourself. Don’t fret; you’re on the right track.
5. Be Positive: While misery loves company, people like and follow positive people. In talking & delegating all of your goals, plans, do so positively. You’re not “fixing a pile of rubbish”, you’re “building an awesome application” or “improving an important machine” or whatever you can say and actually believe when you say it. Eat right, exercise, and/or surround yourself with positive influences. Make this the year you commit suicide, reorganize & clean your working environment/home office, or just accomplish a small goal that’s not work related that you use as positivity fuel. Mentor & encourage your team but don’t come off as a know-it-all/holier-than-thou. Play to win.

Successful Exit

If you know what you/your team needs to be working on today, next month, you’ve approved the hiring you need, and the team is all tasked up, then it’s time to move on.

Notes on Tier #5 – 8

The latter teirs are development specific, but rely on a strong firm foundation from the first 4. Remember, the human component is your weakest link, not your Factories. Strengthen them; do not enter these 4 latter tiers of first 4 are weak.

Tier #5: Build

You need to easily build the software. This is the most common, and frequent, option you’ll do many times a day, every day. The longer this takes, the longer it takes you to develop. This can also kill your team’s momentum. If something goes wrong, such as someone checking in a bug that breaks the build into source control, or a feature needing be implemented for a presentation by some date, a slow build exacerbates time to resolution.

Worse, some developers will start making their own build setup, leading to code that works on some developers machines and not on others. Validation of working code suddenly becomes meaningless which can adversely affect project schedule metrics. Even worse, you can think your code works, but it doesn’t for others. Who’s right?

Milestones

1. Quick: The build needs to run quickly. This obviously an arbitrary metric based on code size, libraries, continuous integration setup, etc. If developers say “the build is slow”, then it’s slow. Utilize libraries so the entire thing doesn’t have to compile, inject dependencies at runtime, utilize modules… whatever it takes.
2. Easy: If the build is complicated to do, you’ll get developers making a lot of mistakes during code validation time before they check in. They’ll either check in bad code, or make their own setup that allows the code to work on only their machine. They should be able to just click 1 button, run an Ant/Maven script, or do a 1 line command line execution.
3. Duplicated: The build should be able to be duplicated on a new developers machine. It’s completely fine to have company standards, such as only Windows 7 PC w/admin rights, Flex SDK 4.6, running Rake 0.8.2. It’s also ok if that takes some time with another developer to setup. What’s not ok is once they do that, it doesn’t work. If there are 3rd party dependencies that cannot effectively be deployed x-machine, find a way to remove them. Solutions include, again, using libraries, modules, development servers vs. local server setup, etc.

Successful Exit

If you can run the build multiple times a day without pulling your hair out, it can be run multiple times on the same code base and not break, and anyone on the team can explain to a new developer 90% of the setup, you’re good to go. Another validation includes Developer A checking in code, then Developer B updating to latest and running the build and it works on their machine as well.

Tier #6: Explosions

When things go awry, you need to immediately know where and why. More than a euphemism for errors/exceptions, explosions also include errors in the application that constantly cause fire drills in the company.

Milestones

1. Exceptions Handled: Handle all errors save null pointers. If you have global exception handling capabilities, implement it/them. This doesn’t mean just wrapping something in a try/catch; it means not only logging the catch, but putting something meaningful in the catch so you’ll understand what it means when it happens.
2. Logging: You need to centralize all long into a single library. It’s ok if this library is your own that builds on top of the built-in provided one, such as Flex’ ILogger. As long as anyone, anywhere on the team wants to log something, they use that vs. print/trace.
3. Visual Log: Optional: A visual window, built into the application itself (SWF/JS, etc), that prints out the log levels. This is done to allow logs to be seen by developers, not users. It also allows non-developers, especially QA, to proactively find problems and talk directly with those responsible vs. the 1st developer they can find. The value this provides in empowerment and time saved should not be underestimated.
4. No More Fire Drills: Wrangling in challenging errors to help prevent fire drills can do 3 things. First, you score a lot of trust in “knowing” why something is wrong vs. 10 people running around aimlessly trying to figure out why. Second, you can clearly cite accountability; is it the server-side team, your team, or the database team? Instead of yelling “I think your stuff broke” you can confidently say it did with helpful data for them to test against. That kind of attitude is much easier to solve problems with. Third, fire drills no longer sabotage your productivity.

Successful Exit

If an error occurs and you know where it happened, a general reason of why, and who’s potentially accountable, you’re in insanely good shape. Users don’t see these errors, just null pointers, or those you’ve built a GUI for (ie Alert error dialogue). Bonus points if fire drills happen on other teams that aren’t related to yours… but you have a way of helping through logging to make testing easier.

Tier #7: Diagnostics

Diagnostics form a range of tools, not just code. They are anything that helps provide insight into code issues or runtime errors without costing more overhead in maintenance than the value they provide. The reason diagnostics is before architecture is that in larger applications, especially in a consulting context, adding simple diagnostics has a ton of value with little effort whereas architecture takes a ton of effort but tends to have low perceived value (yes, it’s valuable, just harder to sell).

Milestones

1. Unit Test Coverage: Even if you’re not doing TDD, unit tests along help find & prevent bugs as well as improve API’s. Unlike code reviews, they don’t get tired, benefit the entire team, and if written well only slightly add to code maintenance. Finally, they can be automated. These are also great things to work on in down times such as when JIRA is cleared, you’ve hit a brick wall with another bug, or are waiting on dependencies for a feature.
2. Integration Tests: These are my favorite on service layers, more so than unit tests. The reason being is most of the Flex projects that I’ve worked on in a consulting context had some form of service layer problem; ie how they talk to the back-end. Both refactoring the service code and writing unit tests that allowed me to automate the testing. It’s also nice to quickly know what part broke with confidence; client, server, or database. Automating these also makes you feel better when the server guys test new code, you can quickly help them test. There are more to integration tests, but these are the low-hanging fruit of ‘em.
3. Automated Builds: Expanding upon Tier #5, streamlining your build process to be more inline with continuous integration practices. This means designating a resource to setup, and maintain, a continuos build tool like CruiseControl/Hudson/TeamCity, etc. You should know who checked in bad code and when.
4. Custom Diagnostics: Some applications have challenging domains of complexity. For example, video; there are TON of things that can go wrong at runtime even if the code compiles just fine. There are also a sequence of events that can cause video to fail, and knowing this sequence is paramount in debugging it, or shrugging it off since there’s nothing you can do (i.e. CDN is down). More importantly, though, sometimes you need a way to glimpse this in a more visual way than a logging window can provide. Conversely, sometimes you need to see environment variables in real-time. This little window can verify your application is running in the mode you think it is for a developer in another country. Simply making a box with a text field in it that showed Flash Player’s sandbox security type saved me 3 days (I lost 3 figuring this out, heh). Additionally, they can help empower others to do testing & diagnostics on their own. Just be careful these are not a security risk for code and/or company information if deployed in production.

Successful Exit

If you have good unit test coverage, can automate their running, your build is automated in some fashion (even if just client side only), and you have gui or command lines way to gain insight to your running application to find out more about errors, you’re in awesome shape.

Tier #8: Architecture

While good architecture requires a book full of information + years of experimentation to get right, what we’re concerned with here is IMPROVING existing architecture. If you are starting a project from scratch and own the architecture, great, use this phase for maintenance. Otherwise, we’re looking for that sweet spot of a 1 day change for a more decoupled, easier to use & test in code system. If something takes longer than 3 days, you’ll be hard pressed to merge it back in and not inconvenience other developers… unless it’s very little code.

Milestones

1. Tight Factories: Any part of your application which takes outside data into it’s system, such as external JSON/XML as well as user input, needs to be further hardened. DRY the code, layer with #moarUnitTests. Form a convention on what happens when parsing/decoding fails; do you throw errors or return null and log them, thus passing the burden of handling null to your system? Decide as a team.
2. Fault Tolerant Services: Whether to a back-end, an external call to JavaScript in a host page, or an ad library… if it’s not your code, it’s a service. Make sure you wrap that with a Proxy that can NOT explode, and logs everything that went wrong. That extra 20 minutes pays off mad dividends.
3. MVC/MVP/MVVM/YourMom/SC/PV/PM: As a team, pick one an over arching architecture pattern and stick with it for new code. I like Supervising Controller, while others of my front-end  ilk like Presentation Model. A lot of JavaScript web frameworks follow more of a Passive View. The only right choice is team consensus & comfortability. Then endeavor as a team to get certain parts moving in that direction. If there is old code, and you’re currently working on it, see if I can convert in less than a day to make more congruent with the rest of the architecture.
4. DRY: If none of the above can be improved, simply making code more DRY is a good fall back task to more unit test coverage.
5. TODO/FIXME/KLUDGE/HACK: Do a find all in the code. If you see those, fix them. #fixBrokenWindows

Successful Exit

You never really exit Tier #8; in fact you tend to loop back to either 7 or 6 and 7 and then back to 8. However, if you find yourself working on new features, or fixing bugs with no other drama and the rest of your team is doing the same… then you’re in epic shape.

Conclusions

As you can hopefully see, my priority list helps keep you focused on delivering value to those that matter quickly. Good software development is iterative. Earning trust is also iterative; you don’t take people’s money and disappear for 6 months (that’s called good Capitalism via Waterfall). This whole list, while referring to software development, is all about relationships with people. Excluding code review, your challenge as a leader is delegating the tasks in each tier to divide & conquer, mitigating conflicts, and solving hard software problems in addition the rest of your responsibilities.

Each tier builds upon the former all the way down the line. While deep in the bowels of fixing explosions in Tier #6, I’m still continuing to build meaningful relationships in Tier #3.

Remember:

1. Report: Know your true boss and what will make her happy.
2. Understanding: Listen, learn about the company, the people, and dive into reading the code.
3. Trust: Earn trust by providing immediate value.
4. Lead: Take the initiative and move the team forward. Delegate, divide, conquer.
5. Build: Make the build fast, easy, and capable of being duplicated if need be.
6. Explosions: When things go awry, you need to immediately know where and why. Prevent fire drills.
7. Diagnostics: Get good unit test coverage, integration tests, automate your builds, and build any custom diagnostic tools needed.
8. Architecture: Decide on a direction/framework, and continue to nurture the code, and developers, in that direction. Loop back to 6 & 7 if need be.

Title photo by Ewan Monro is under a Creative Commons – ShareAlike Attribution license.

Refactoring is the discipline of applying a multitude of small, low-risk techniques to a code base in order fix and improve it. While corroborated by the software community, employing such techniques in a consulting context can be challenging because people are involved, often in a negative situation.

You need 2 weapons to win the refactoring battle. First, you need to understand what refactoring techniques are at your disposal and how to implement them. Below, I’ve listed the core ones I see needed time and time again. Second, you need to have a plan on how you engage the client company and their employees to allow you to do the first. That’s covered in the next article.

I’ve talked about how you do various parts of these initiatives in previous articles. What I want to talk about today is 1 of the 2 things: The core things you need to be concerned about in refactoring. In the next article we’ll talk about a prioritized plan you should follow when engaging a client where refactoring is needed. These are aimed at being used in the worst situations, but can provide positive benefits in benign or benevolent situations as well.

You should be able to take this blog post and the next, and from them both progressively improve a poor quality code base as well as engage a challenging client amongst challenging circumstances.

Note On Languages

I’ve intermingled JavaScript, ActionScript, and Lua to help provide a variety of examples, show this list is technology agnostic, and provide corroboration of techniques across technologies. That and I miss coding Lua since I’ve been so busy and needed an excuse. This does have a strong focus with front-end languages, namely those which run in a browser.

Nomenclature

Factory/Decoder/Parsing: Taking any kind of data, such as JSON or XML, and converting  into something you need it for. This could be as simple as accessing properties on it to converting it to a completely different type of object, or even creating new objects from it.

Coding Conventions

• Factory Errors vs. Null: Below, I’m using both error and null. Error means if a Factory/function/decoder fails in anyway to understand its data, you throw an error. This helps in unit testing, and expose errors early in the weakest part of a front end application; those that utilize non-strongly typed data. Optionally, when you find a parsing error, you return null instead. Consumers of the Factory must test for null, but you mitigate runtime exceptions and expose faulty consumers. I like using both, but it’s a lot of work to be sure.
• Don’t Create Grenades: Regardless of language, you shouldn’t create grenades. This means don’t use throw/error. Instead, dispatch a custom error event that is opt-in. Consumers shouldn’t have to worry about the code they’re using bubbling up problems to their area of worry unless they’ve opted into those concerns. ActionScript, JavaScript, and Lua also don’t have a strongly typed ways to verify with the compiler that these contracts have been met, unlike Java’s throwable. Thus, don’t throw, inform to those who wish to know. Log regardless.
• Factory vs. VO Self Parsing: Some like Factory’s to parse external data (JSON/XML, etc.) into ValueObjects for an increased likehood of DRY code (shared parsing code that’s easier to unit test). Others like ValueObjects to serialize/deserialize themselves to reduce coding overhead as well as make the API easier to use. I’ve used both below, but I’ve found this is usually a team decision and also depends on how complex your VO’s are.
• Model vs. Command vs. Remote Proxy: From architecture perspective, some people prefer to have a Model handle Application Data, and a Command/Controller/Presenter handle remote Service calls to update that data. Some prefer wrapping both inside of a Remote Proxy (basically a Model that has serivce(s) inside it), and allowing Controllers/Mediators/Presenters/ViewControllers to make calls on those Remote Proxies. I use both Model & Commands below, but no Remote Proxies. It’s up to you and your team which you think is best. If you’ve gotten this far in the discussion, you’re clearly better off than most. The point here is solve for Model and/or Application logic encapsulation.
• try/catch: I use try/catch liberally. Developers concerned about performance, such as game developers or those using complicated parsing algorithms should take note the significant performance cost of try/catch as well as liberal use of ExtractMethod. This article assumes the code is in bad shape and you/your team needs to refactor it over many months to ensure your team can prevent fire drills, meet milestones, and start to build a firm foundation. If the code base has the following problems outlined in this article, your first order of business should be fixing them first before you start optimizing. If you really need performance, just use unit tests instead and pray.
• Singletons: While not a replacement for global variables, and generally frowned upon, well written ones are a good first step if they have access control and utilize strong-typing. (ex Model).

Refactoring Basics

Improving existing code can be done in 3 ways: handling exceptions, using a good architecture, using strong-typing if available, and following refactoring paths. The reason you use refactoring, especially in a consulting/contracting context with existing code bases is that you’re not allowed to rewrite them from scratch. Thus an unwritten bequeathment of responsibility occurs between the code base and you: it now becomes your problem. Its failings become your failings. The insecurity it gives your client now has them project that insecurity, and responsibility to remedy it, onto you.

I’m assuming you’ve already done a code review by this point to ensure you, or your firm, want to even get involved. If you don’t but are still here, it’s probably because they’re paying you a lot of money to wallow in their mess. In that case, welcome to the suck.

The original definition of refactoring, referring to Martin Fowlers here, is making a small, positive change that has low inherent risk. This is done to ensure you don’t break anything. The theory is, over time if you make enough positive changes, you’ll make the whole code base positive.

Life is more complex than that, but the point here is that you use refactoring to affect positive change without breaking the code and looking like an idiot in front of your client and ticking off their customers. You do not make large, sweeping changes to the code base, even if they are needed. Take it slow and steady, 1 little change at a time. Some of the examples below are small (such as using strong-typing in place of Object and *), and some are larger (such as modifying your Model layer) are larger. Sometimes such changes you need help in tackling because it’s sphaghetti code; team refactoring can be invigorating and you get a lot more done, so don’t think you have to do this on your own.

When in doubt, make a small, positive change. If you take more than 3 days to make a change, abort.

Exceptions

Your first priority in refactoring should be handling exceptions or errors. It’s ok if you see how sausage is made in the meat factory, but it’s not ok if your client does. You can turn explosive bedlam into a controlled spectacle. That’s right, things breaking can be made into looking like your in total control. Magic isn’t magic; it’s sleight of hand through distraction.

“Look at this pretty logging window created by our company to handle such situations. All errors are now known to us, when and where they happen. This awesome ability to report said errors amongst your staff, not just QA, but you, your customers… everyone, even while in production! No surprises or fire drills here… ALL at no extra charge.”

But how do we get there? Let’s start at bottom.

A lot of programming languages execute functions or methods in 2 modes: protected or omg-it-might-blow-up-we’re-hardcore mode. JavaScript/CoffeeScript, ActionScript, and Java for example use try/catch. Ruby uses begin & rescue. Python uses try/except. Lua uses pcall. Most languages have various ways to ensure that if code explodes, it’s done so in a controlled fashion. Additionally, some have global exception handling as well.

Exceptions are the first area of uncontrolled chaos that must be contained. No controlled refactoring can take place without tackling these areas first.

Architecture

Architecture comes into play for large code bases, although, using it for small code bases can help you learn (albeit hard to appreciate as it’ll often get in your way of getting r done).

Architecture refers to how an application is built. It’s usually an amalgamation of design patterns used in a common way. For the sake of this article, we’re using architecture in a small sense of the word in helping move parts of the code to be more encapsulated, make things that break easier to identify, test in isolation, that sort of thing. While references to MVC/MVP/MVVM can get really esoteric and subjective, we’re not concerned with that.

What you need to verify with defining “good architecture” for you and your team is how it works, agree on approach, and ensure it helps you test things in isolation. If y’all agree, can be independently productive, and if something breaks you know where… that’s good architecture. Don’t get hung up on the definitions, get hung up on the contract on how things talk to each other, and that your team all agrees on that contract. This’ll come into play later in the next article regarding team synergy.

Strong Typing

Not much to say here. Don’t use Object/*/Dictionary. If you do, wrap it with a strongly typed API or unit tests. Putting the burden on the consumer of those objects leads to lots of code. Lots of code == bad.

Don’t do this:

var person:Object = {};
person.firstName = "Jesse";
person.lastName = "Warden";

Do this:

class PersonVO
{
   public var firstName:String;
   public var lastName:String;
}
 
var person:PersonVO = new PersonVO();
person.firstName = "Jesse";
person.lastName = "Warden";

When you don’t have strong-typing (JavaScript/CoffeeScript/Lua), #moarUnitTests. For JavaScript use jshint, jslint, and annotations that various frameworks use to infer type like Google’s Closure compiler for great justice.

Refactoring Paths

Beyond the suggestions below + mentioned resources, you have to use your own intuition. Refactorings on their own are good but when done as a team effort in strategic areas, you can make greater headway, earn more trust from your client, and do more “relevant” refactoring.

There are some things you’ll know are right such as investing in a good GUI logger, making friends with IT, etc. Others, on what to fix first, how to support dual architectures in some areas, and how to portray transparency to your client… some of those are just experience. When in doubt, go with your gut. The less code of the choices, the better.

A good refactoring path allows you the ability to code right now, satisfy non-programmers for trust earning, with assurance you haven’t coded yourself into a corner. It’s like a good marriage: compromise.

You’ll lose some battles. The goal is to win the war.

You’ll have detractors such as who don’t have their Code License; knowing the rules before breaking them… or some who are just afraid. Some are ivory tower zealots who aren’t punished for getting nothing done (some are rewarded). Some have emotional scars from attempting to fix things, and are hesitant to do what’s right. This is usually when they know the value but cannot effectively articulate it to stakeholders (this is called a “software developer”). Some are just insecure and don’t believe in their own refactoring ability and thus hold you back as well (low D on DISC). Some think you’re full of it.

Refactoring is temporal. You are creating a positive change to last for awhile… but not forever. It’s ok to refactor something, then refactor it again later as long as you’re actually making an improvement upon a problem. If a ship is taking on water, the first thing you do is plug the holes with wooden spikes. Later, when the French stop firing at you and your hangover is gone from drinking all the wine you found on their now apprehended ship, you can plug the holes proper with a leather & tar mixture. Then once in dry dock, replace the planks.

Architecture is not temporal… but can be as well. You’re supposed to establish rules and guidelines on how you approach a code base. Sometimes, however, you can jury-rig a solution to hit some deadline, or get a build out the door for some managerial milestone. Later, you can re-assess. Sometimes, you’re initial hack is controlled; meaning you are writing throwaway code on purpose. In doing so, you can make it easier to fix later.

Architecture is generally decided upon by the team, and you move forward for the length of the project on it.

Refactoring is a strategy that can change daily.

Step #1: Exceptions

Case: Service layer has 3 failure points: entry, error, and exit parsing.
Solution: Validate post data, log error, wrap parsing/factory/decoding.
Language: JavaScript using AmplifyJS

Bad Code:

amplify.request.decoders.appEnvelope =
    function ( data, status, xhr, success, error )
    {
        if ( data.status === "success" )
        {
            success( data.data );
        } else if ( data.status === "fail" || data.status === "error" )
        {
            error( data.message, data.status );
        } else {
            error( data.message , "fatal" );
        }
    };
 
amplify.request.define( "decoderExample", "ajax", {
    url: "http://server.com/service/doSomething/",
    type: "POST",
    decoder: "appEnvelope"
});
 
amplify.request({
    resourceId: "decoderExample",
    success: function( data )
    {
        data.foo; // bar
    },
    error: function( message, level )
    {
        alert( "always handle errors with alerts." );
    }
});

Refactored Code:

/*
Our decoder function is now:
1. shielded from bad data
2. assumes the response not well formed from the get go
3. logs each parsing error separately for easier diagnosis
*/
 
// modified appEnvelope decoder function
function ( data, status, xhr, success, error )
{
    try
    {
        if(data == null || data == "")
        {
            error("Decoding error in parsing data.");
        }
        else if(data.status == null || data.status != "")
        {
            error("Unknown status.");
        }
        else if(data.message == null || data.message != "")
        {
            error("Uknown message.");
        }
        else if ( data.status === "success" )
        {
            success( data.data );
        }
        else if ( data.status === "fail" || data.status === "error" )
        {
            error( data.message, data.status );
        }
        else
        {
            error( data.message , "fatal" );
        }
    }
    catch(e)
    {
       error("Decoding error.");
    }
};
 
/*
Notice we:
1. Validate our requestData (you could do more than a null check) before making an actual service call.
2. Additionally, we log the failed request with the Service name and reason for easier diagnosis in a larger application.
*/
 
if(requestData != null)
{
    amplify.request({
        resourceId: "decoderExample",,
        data: requestData,
        success: function( data )
        {
            data.foo; // bar
        },
        /*
        Our request error handling now has:
        1. a log, vs. an alert, for logging our error
        2. a common logging signature for easier diagnosis in a larger application
        */
        error: function( message, level )
        {
            console.error("MyService::error, message: " + message + ", level: " + level);
        }
    });
}
else
{
    console.error("MyService::error, requestData is null.");
}

Case: Null values in GUI. View/GUI must assume data is not always good; must handle null without exploding.
Solution: Check for null using if/then.
Language: ActionScript 3

Bad Code:

myTextField.text = person.name;

Refactored Code:

/*
We now:
1. check for a null VO first
2. if it is in fact null, we have a visual, conventional way of displaying null values w/o exploding.
3. if a property of the VO is null, that'll display as normal (in this case as an empty String)
*/
 
if(person)
{
   myTextField.text = person.name;
}
else
{
   myTextField.text = "???";
}

Case: Global Exception Handling
Solution: Add global handler that logs errors away from users to see.
Language: ActionScript and JavaScript

Refactored ActionScript Code:

package
{
   import flash.display.Sprite;
   import flash.events.Event;
   import flash.events.UncaughtErrorEvent;
 
   public class ErrorHandling extends Sprite
   {
      public function ErrorHandling()
      {
         super();
         addEventListener(Event.ADDED_TO_STAGE, onAdded);
      }
 
      private function onAdded(event:Event):void
      {
         loaderInfo.uncaughtErrorEvents.addEventListener(UncaughtErrorEvent.UNCAUGHT_ERROR, onUncaughtError);
      }
 
      private function onUncaughtError(event:UncaughtErrorEvent):void
      {
         trace("Uncaught Error: " + event);
         try
         {
            // note, fails in Asyncronous errors
            throw new Error("stack trace creator");
         }
         catch(error:Error)
         {
            trace(error.getStackTrace());
         }
      }
   }
}

Refactored JavaScript Code:

// via http://stackoverflow.com/questions/951791/javascript-global-error-handling
//
// more techniques here for Opera/Safari older versions
// http://stackoverflow.com/questions/645840/mimic-window-onerror-in-opera-using-javascript
//
// also note various browsers pass different arguments, and some suggest
// handling existing handlers if they exist
// https://developer.mozilla.org/en/DOM/window.onerror
//
// also note that the error in some browsers will still occur unless you handle it
// a certain way; here we're returning true from the error handler to do so.
 
window.onerror = function()
{
    if(arguments && arguments.length && arguments.length > 0)
    {
        var logStr = "";
        var len = arguments.length;
        for(var index = 0; index < len; index++)
        {
            logStr += "\t" + arguments[index] + "\n";
        }
        console.error("Global Error:\n" + logStr);
    }
    else
    {
        console.log("Global Error");
    }
    return true;
};

Step #2: Architecture

Case: Encapsulation of statefull, Model data.
Solution: Encapsulate Setting of Model Data to single place (DRY)
Language: Lua in Corona SDK

Bad Code:

_G.score = 0
 
local scoreText = display.newText()
function updateScoreText()
   scoreText.text = _G.score
end
updateScoreText()
 
local powerUp = display.newImage("powerup.png")
powerUp.name = "powerUp"
 
local enemy = display.newImage("enemy.png")
enemy.name = "enemy"
function enemy:collision(event)
   if event.phase == "began" then
      if event.target.name == "bullet" then
         _G.score = _G.score + 1
      end
   end
end
enemy:addEventListener("collision", enemy)
 
local ship = display.newImage("ship.png")
function ship:collision(event)
   if event.phase == "began" then
      if event.target.name == "powerUp" then
         _G.score = _G.score + 1
         updateScoreText()
         return true
      end
   end
end
ship:addEventListener("collision", ship)
 
function onTouch(event)
   if event.phase == "began" then
      local bullet = display.newImage("bullet.png")
      bullet.name = "bullet"
      return true
   end
end
 
Runtime:addEventListener("touch", onTouch)

Refactored Code Round 1:

--[[
Notice we've added an addScore method to:
1. have a central place everyone can update the model data
2. make it easier to identify WHO is calling this add function
3. ensure we never forget to update our Text Field with the latest value like in the original code
]]--
 
function addScore(amount)
   _G.score = _G.score + amount
   updateScoreText()
end
 
-- Fixing our enemy collision handler to now call our encapsulated method
function enemy:collision(event)
   if event.phase == "began" then
      if event.target.name == "bullet" then
         addScore(1)
      end
   end
end

Refactored Code Round 2:

--[[
Now we use a more generalized Observer pattern, and dispatch an event
when the value changes. The system is free to listen to this event
and react however it needs to. This allows the GUI to change w/o worrying
updating the model changing behavior to compensate for a changing GUI API.
]]--
 
function addScore(amount)
   _G.score = _G.score + amount
   Runtime:dispatchEvent({name="onScoreChanged", newScore=_G.score})
end
 
-- we now can update more GUI parts as they become needed
function onScoreUpdated(event)
   updateScoreText(event.newScore)
end
Runtime:addEventListener("onScoreChanged", onScoreUpdated)
 
-- notice this includes encapsulated GUI controls as well;
-- they can just listen in on the Event Bus.
-- Here, we have a Level Progress bar that listens to the score change
-- to visually indicate how far along the user is until they level up.
require "com.company.project.controls.ProgressBar"
 
LevelUpProgressBar = {}
 
function LevelUpProgressBar:new()
 
   local bar = ProgressBar:new({width = 100, height = 60})
   bar.levelMax = 1
 
   function bar:onScoreChanged(event)
      self:setProgress(event.newScore, self.levelMax)
   end
 
   function bar:onLevelMaxChanged(event)
      self.levelMax = event.newLevelMax
   end
 
   Runtime:addEventListener("onLevelMaxChanged", bar)
   Runtime:addEventListener("onScoreChanged", bar)
 
end
 
return LevelUpProgressBar

Case: Access Control for Statefull Model Data Using Application Logic (English: you need to update some model data, and multiple parts in your application need to do this). aka, removing global variables.
Solution: Utilize Proxy Pattern or Command Pattern for single access (DRY), this ensures there is only 1 way to update your model data, inputs are validated. This makes it easier to debug who is calling/invoking it, and what data they are passing, as well as ensuring those who need to know about the change are informed.
Language: ActionScript

Bad Code:

var firstName:String             = firstNameTextInput.text;
var lastName:String              = lastNameTextInput.text;
var birthdate:Date               = birthdateControl.value;
var myPersonVO:PersonVO          = new PersonVO(firstName, lastName, birthdate);
MyStaticModel.person             = myPersonVO;
MyStaticModel.person.lastName    = "McTaggart"

Refactored Code:

// First we create a factory class to test our creation & validation in isolation.
// This is also easier to write unit tests for. It's also DRY in that all classes
// who need to create PersonVO's and validate them are using the same code to do so.
 
class PersonFactory
{
   public static function getPerson(firstName:String, lastName:String, birthdate:Date):PersonVO
   {
      if(firstName == null || firstName == "")
      {
         throw new Error("firstName cannot be null nor an empty String.");
         return;
      }
 
      if(lastName == null || lastName == "")
      {
         throw new Error("lastName cannot be null nor an empty String.");
         return;
      }
 
      if(birthdate == null)
      {
         throw new Date("birthdate cannot be null.");
      }
 
      return new PersonVO(firstName, lastName, birthdate);
   }
 
   public static const validateLastName(lastName:String):Boolean
   {
      if(lastName != null && lastName != "")
      {
         return true;
      }
      else
      {
         return false;
      }
   }
}
 
// Second step, create a change event that those who need to know about a changing
// personVO in the model can be made aware of.
 
class PersonModelEvent extends Event
{
 
   public static const PERSON_CHANGED:String = "personChanged";
 
   public function PersonModelEvent(type:String)
   {
      super(type, false, true);
   }
 
}
 
/*
Third, create a Model class Singleton that:
- dispatches change events when it's internal person variable is changed, or modified
through accessor controls
- validates creation & changes
NOTE: If classes need to operate on PersonVO itself, create additional methods to
operate on the internal PersonVO to ensure proper change events are dispatched.
NOTE: For those who wish to remove Singletons, create as an instance, and inject the dependencies,
whether via a DI framework such as SwiftSuspenders/Guice/Spring, or manually via
getter/setters for those classes who need instance access.
*/
 
class PersonModel extends EventDispatcher
{
 
   private var _person:PersonVO;
 
   public function get person():PersonVO { return _person; }
   public function set person(value:PersonVO):void
   {
      _person = value;
      dispatchEvent(new PersonModelEvent(PersonModelEvent.PERSON_CHANGED));
   }
 
   public function PersonModel()
   {
   }
 
   public function setNewPerson(firstName:String, lastName:String, birthdate:Date):void
   {
      // first, validate data is good, don't want to pollute our model
      var personVO:PersonVO;
      try
      {
         personVO = PersonFactory.getPerson(firstName, lastName, birthdate);
         person = personVO;
      }
      catch(error:Error)
      {
         trace("PersonModel::setNewPerson error: " + error);
      }
   }
 
   public function updateLastName(lastName:String):void
   {
      if(person)
      {
         if(PersonFactory.validateLastName(lastName) == true)
         {
            person.lastName = lastName;
            dispatchEvent(new PersonModelEvent(PersonModelEvent.PERSON_CHANGED));
         }
         else
         {
            trace("PersonModel::updateLastName, failed because the lastName '" + lastName + "' is invalid.");
         }
      }
      else
      {
         trace("PersonModel::updateLastName, failed because the current person is null.");
      }
   }
 
   private static var _inst:PersonModel;
 
   public static function get instance():PersonModel
   {
      if(_inst == null)
         _inst = new PersonModel();
 
      return _inst;
   }
}

Refactored Code 2:

/*
In the case of need of many needing to affect the Model, and there is no
desire to all Models/Proxies to have public methods accessible by others,
you can use the Command pattern. Additionally, this solves the use case
where multiple Models are needed to be accessed and/or modified. Finally,
we also handle asynchronous use cases where you need to update a Model(s)
based on a server response. Notice how each step is validated, and a log is
produced if any step fails. You need to know WHERE it failed and WHY.
Also, no rollback support in the following.
*/
 
class CreatePersonEvent extends Event
{
   public static const CREATE_PERSON:String = "createPerson";
 
   public var firstName:String;
   public var lastName:String;
   public var birthdate:Date;
   public var personModel:PersonModel;
   public var organizationModel:OrganizationModel;
 
   public function CreatePersonEvent()
   {
      super(CREATE_PERSON);
   }
}
 
class CreatePersonCommand extends AsynCommand
{
 
   // injected dependencies; you could use a DI framework,
   // wrap Command class creation with these, or simply pass
   // in through the Event payload like the below shows.
   private var personModel:PersonModel;
   private var organizationModel:PersonModel;
   private var createPersonService:RESTFullCreatePersonService;
   private var personVOToCreate:PersonVO;
 
   // Create a PersonVO, set on the PersonModel,
   // and ensure the OrganizationModel is updated with the
   // new person once the server has verified it's creation.
   public function execute(event:CreatePersonEvent):void
   {
      try
      {
         personModel = event.personModel;
         organizationModel = event.organizationModel;
         personVOToCreate = PersonFactory.getPerson(firstName, lastName, birthdate);
         if(personVOToCreate != null)
         {
            createPerson()
         }
         else
         {
            throw new Error("ERROR: CreatePersonCommand::exeucte, PersonFactory returned null.");
         }
      }
      catch(error:Error)
      {
         trace("ERROR: CreatePersonCommand::execute, error: " + error);
         finish();
      }
   }
 
   private function createPerson():void
   {
      createPersonService = new RESTFullCreatePersonService();
      createPersonService.addEventListener("success", onCreatePersonSuccess);
      createPersonService.addEventListener("error", onCreatePersonError);
      createPersonService.createPerson(personVOToCreate);
   }
 
   private function onCreatePersonError(event:Event):void
   {
      trace("ERROR: CreatePersonCommand::onCreatePersonError, event: " + event);
      finish();
   }
 
   private function onCreatePersonSuccess(event:Event):void
   {
      // server has successfully created PersonVO, let's update ours with what
      // the server sent back to ensure we now have a proper PersonVO.ID set
      // by the server
      if(createPersonService.savedPersonVO != null)
      {
         personModel.person = createPersonService.savedPersonVO;
         organizationModel.addNewPerson(personModel.person);
      }
      else
      {
         trace("ERROR: CreatePersonCommand::onCreatePersonSuccess failed, server sent back a null savedPersonVO.");
      }
      finish();
   }
 
}

Step #3: Strong Typing

Case: Using loose typing/variants/Objects/*, resulting in loss of compiler time checking and thus errors that only appear at runtime.
Solution: Use strong-typing.
Language: ActionScript

Bad Code:

/*
Notice:
- personVO.lstName is a mispelling, but no error will be raised by the compiler.
- no error will be raised by the updatePerson function as Object is dynamic,
and lastName will be created and set on the personVO Object.
- no error will be raised if you pass something other than a personVO to updatePerson
except for runtime errors
*/
 
var personVO:Object = {};
personVO.firstName = "Jesse";
personVO.lstName = "Warden";
 
function updatePerson(person:*, newNameFromStringBlock:String):void
{
   var nameArray:Array = newNameFromStringBlock.split(" ");
   person.firstName = nameArray[0];
   person.lastName = nameArray[1];
}

Refactored Code:

/*
Notice:
- our "lstName" will be caught by the compiler
- our updatePerson function ensures only a PersonVO is accepted to be operated on. If not,
our compiler will let us know.
*/
 
class PersonVO
{
   public var firstName:String;
   public var lastName:String;
}
 
var personVO:PersonVO = new PersonVO();
personVO.firstName = "Jesse";
personVO.lstName = "Warden";
 
function updatePerson(person:PersonVO, newNameFromStringBlock:String):void
{
   var nameArray:Array = newNameFromStringBlock.split(" ");
   person.firstName = nameArray[0];
   person.lastName = nameArray[1];
}

Case: Service layer has no input validation, is hard to test, and has no error reporting. Burden is on class user to wrap in try/catch as well as parse server response including knowing business logic. Finally, asynchronous loader is not a class member variable, thus potential memory leak + harder to debug.
Solution: Create a Service class that is easy to use, validates inputs, and logs errors. Encapsulates business logic.
Langauge: ActionScript

Bad Code:

class CreatePersonService extends EventDispatcher
{
 
   public var loader:URLLoader;
 
   public function CreatePersonService()
   {
      super();
   }
 
   public function createPerson(person:PersonVO):void
   {
      var loader = new URLLoader();
      loader.addEventListener(Event.COMPLETE, onComplete);
      var req:URLRequest = new URLRequest("http://someserver.com/restful/api/");
      req.data = person.toJSON();
      loader.load(req);
   }
 
   private function onComplete(event:Event):void
   {
      dispatchEvent(new Event("success"));
   }
}
 
var service:CreatePersonService = new CreatePersonService();
service.addEventListener("success", onSuccess);
 
function onSuccess(event:Event):void
{
   var json:Object = JSON.decode(service.loader.data);
   var person:PersonVO = new PersonVO(json.firstName, json.lastName, json.birthdate);
}

Refactored Code:

/*
Note:
- createPerson validates you gave it a valid PersonVO.
- all errors within class are opt-in, no throw's. User has to register event listener to hear; #dontCreateGrenades
- class assumes single-concurrency if possible, kills previous load call before proceeding
- listens for all errors, and responds with 1 error with more detail logged if needed; simplifies API
- wraps actual call to verify SecurityError isn't thrown and if it is, it's logged w/ opt-in error
- uses united tested Factory for DRY parsing and assumes
- assumes Factory can fail, and reports it as such
- result in the form of a strongly-typed ValueObject in a read-only property to ensure proper API usage
*/
class CreatePersonService extends EventDispatcher
{
 
   private var _savedPersonVO:PersonVO;
   private var loader:URLLoader;
 
   public function get savedPersonVO():PersonVO { return _savedPerson; }
 
   public function CreatePersonService()
   {
      super();
   }
 
   public function createPerson(person:PersonVO):void
   {
      _savedPersonVO = null;
 
      if(person == null)
      {
         onError("You cannot pass in a null person to create.");
         return;
      }
 
      if(loader == null)
      {
         loader = new URLLoader();
         loader.addEventListener(IOErrorEvent.IO_ERROR, onIOError);
         loader.addEventListener(SecurityErrorEvent.SECURITY_ERROR, onSecurityError);
         loader.addEventListener(Event.COMPLETE, onComplete);
      }
      else
      {
         try
         {
            loader.close();
         }
         catch(error:Error)
         {
            // only exception that is ok to swallow
         }
      }
 
      try
      {
         var req:URLRequest = new URLRequest("http://someserver.com/restful/api/");
         req.data = person.toJSON();
         loader.load(req);
      }
      catch(error:Error)
      {
         onError("CreatePersonService::createPerson, error: " + error);
      }
   }
 
   private function onIOError(event:IOErrorEvent):void
   {
      onError(event.text);
   }
 
   private function onSecurityError(event:SecurityErrorEvent):void
   {
      onError(event.text);
   }
 
   private function onComplete(event:Event):void
   {
      try
      {
         _savedPersonVO = PersonFactory.parsePersonFromJSON(loader.data);
         if(_savedPersonVO != null)
         {
            dispatchEvent(new Event("success"));
         }
         else
         {
            onError("CreatePersonService::onComplete, factory returned a null PersonVO.");
         }
      }
      catch(error:Error)
      {
         onError("CreatePersonService::onComplete, parsing error: " + error);
      }
   }
 
   private function onError(errorString:String):void
   {
      trace("ERROR: CreatePersonService::onError: " + errorString);
      dispatchEvent(new Event("error"));
   }
}

Step #4: Fine Tuning

If you made it this far, and have managed to continue to maintain the above, congratulations! This stuff is icing on the cake. You can continue onto smaller scope refactorings, as well as continuing to increase your unit test coverage.

Final Tips

1. When casting, ensure result is not null before using. If null, log the casting failure.
2. For Flash Player, set ExternalInterface.marshallExceptions = true if using ExternalInterface
3. Log all errors
4. Code Review. Often. Weekly. Whatever, pick a schedule.
5. Object / * / Dictionary == if you can’t avoid, wrap with unit tests
6. Remember: small, low-risk changes.
7. TODO was invented for a reason.
8. If you haven’t committed to source control for 3 full days of working, abort and just check into a branch for later revisiting/reference.

Conclusions

Remember, the most important refactoring you can possibly do is reading the code base for 1 hour a day, no more. Just 1 person. This is has more positive impact than good unit test coverage. Daily code reviews, even in isolation, are extremely important. The earlier in the project life cycle, the better.

From that, you can make small, low risk changes. Keep track with TODO’s. Breathe, Rome wasn’t fixed in a day, and a pile of rubbish wasn’t fixed in one either.

For further, positive refactorings you can check an older list by Martin Fowler, and this book has a ton of wonderful ones + tests as well. While a Code Review has decreasing value the more people you add in terms of defects found, the ability to learn & make positive suggestions for architecture are wonderful. If you’re alone, use codereview.stackexchange.com.

In the next article I’ll show you how to get help with the above… and permission.

Why P90X?
I’m a programmer who’s a work-a-holic so I sit for 12+ hours a day, 7 days a week. Recently, my part of the tech industry has gone into major upheaval (change is a constant, but this was crazy), and the only thing I can control about it is my well being to adapt to the change. I always found exercise to be a positive outlet; strength training for the frustration when code doesn’t work or a client drives me bonkers, and cardio for keeping my body warm and not achy from sitting so long. My business partner is a meat head and always raves about P90X that he and his wife do together, so I thought I’d give it a try.

Personal/Life Challenges

• I hadn’t worked out in 4 years (used to do strength training + too much cardio 3 times a week). I couldn’t even pass the test they suggest you pass before you start P90X, but I forged ahead anyway. #growl
• I run my own business so even if I only bill for 5 hours in the day, I still work 14. Being in software, you have to constantly research/read/experiment to stay up to date; this isn’t an option, you just have to do it whether you want to or not (most of us do via our short attention spans). I have to do this in addition to everything else required to run a small business (taxes, paperwork, lead generation, dealing with insane clients, etc). As such, time is in short supply, and valuable, for me.
• I have 2 young kids who are spaztastic like me.
• I’m a beer snob. I like trying new beers every few weeks, from light to dark, foreign and domestic. Water weight.
• I’m not attempting to do the P90X diet. I want to, but not for round 1.
• I’m taking zero supplements, nor shakes. While I’ve found positive results from protein bars when I was younger, nothing of the sort here.
• I’m “not 23″, lulz. I’m 32. My metabolism is not what it was at 18 – 24 where I could eat 10 billion things and show nothing.

Results

So what’s positive that’s happened 3 weeks in?

1. I found some “brand new” jeans from 4 years ago that I could fit into (32/32 vs 34/32). I wasn’t really trying to lose weight, just feel better and bulk up. 3 weeks in, though, clearly my beer stocked stomach is showing results.
2. The first week I couldn’t even use weights. Week 3 I’ve maxed out to the 27lb band (red one) for most work outs, falling back to the black one for some of the harder arm/leg exercises. I’m already having to order heavier bands and maybe a few free weights for some of the leg exercises I’d rather use dumbbells for.
3. Week 1 and 2, I couldn’t finish Kenpo X. Today, I did and only did 18 of the 20 reps on 2 of the exercises. Next time, I know for a fact I’ll finish no problem.
4. First time Yoga, I couldn’t do half of the poses. Now, I’m just getting my tail kicked by Warrior 3 and Half-No-One-Is-Supposed-To-Bend-That-Way-Moon. While all my poses aren’t perfect, it’s insane how quickly I got better at doing + holding just 3 weeks in. I’m uber paranoid about hurting myself so am pretty impressed.
5. Anytime a stretch required me to stand, lean to the floor with my hands and touch it, I’d get my hands to my knees. Now I can touch the floor. 2 years of working out on my own I never was able to do that; it shows just how much repetition + following P90X’s lead can really make you more flexible.
6. I only got a heart rate monitor (+ app + connector) 2nd week in, but already it’s a lot easier to keep my heart rate in the zone. Yes, I do try to raise it, but I’m just amazed at how good I feel even in Zone 3 and 4… (yes, was on the floor week 1 and 2 breathing heavily, not moving).
7. I could only do 1 pull up the entire workout week 1. Using the bands + chair combo sometimes, I can do a lot more, improving week to week. I’m still not happy with my current state, but looking at the worksheets, I’ve seriously improved. I thought it’d take 2 months, but apparently it doesn’t.
8. As I slouch a variety of ways, whether at my desk or at Starbucks (other office), my legs will sometimes fall asleep if I get lost in the code. It used to take 15 minutes for an asleep limb to stop being in pain and wake up. Now it takes 15 seconds. I actually made my leg fall asleep on purpose the other day because I thought it was a fluke. That took me 2 1/2 months to get to back when I was working out 3 days a week even with a personal trainer (who cost bling I might add).

…the other positive thing that’s happened, and I didn’t intend for this, is that some of my friends who were doing P90X, unbeknownst to me, have being inspired to bring it more. Others who were on the fence about trying it have said they’d look into it. Two unrelated people have said the same thing: “The DVD’s have been on my shelf, un-opened for awhile… maybe I’ll try it next Monday.”

Remember folks, this scrawny, beer bellied work-a-holic programmer is getting results in 3 weeks in, feels better emotionally, and I’m only doing 80% of my physical best because of my lack of previous fitness + no good diet + 2 year old hugging me during leg squats + 4 year old falling on me during Yoga + her majesty demanding I do more cardio in addition to my regular P90X schedule. I’ve even switched to more wine vs. beer just because I don’t want to ruin this momentum.

Finally, if you’re still not losing enough fat to visibly see a difference, reading this thread on another forum helped a few people I know feel better.

So good luck and continue to bring it!

It took me 4 days to build an Elevator via Box2D in Corona SDK. I wrote the actual code and API for it in about 30 minutes. Getting the physics to work proved way more challenging than I anticipated. I’m still learning the ins and outs of Box2D as it relates to game development, so wanted to report on how I went about it and what I learned.

The following article talks about how I’m using Box2D in my game, the 4 approaches I took to building an elevator, and what I eventually decided upon.

Please keep in mind some of the things below are related to Corona SDK’s implementation of Box2D. Thus, some of the issues and compensation measures I took may not need to be done, nor happen, in other implementations and platforms.

Why An Elevator?

Elevators provide an interesting transition in both games and movies. They also provide easier ways for game developers to unload and load a level, provide a rest for the player, and/or make it easy to provide cut scene dialogue to prepare for the player for an upcoming part. In a lot of horror movies, they actually are the focal point of a lot of scary scenes such as Resident Evil, and also provide a perfect setting for comedic relief such as such as Spiderman 2 and Deep Rising (good ole Ipanema).

One of my favorite easter eggs was from Max Payne was in an elevator. You walk into a shoddy NYC apartment building elevator alone, with the notorious off-kilter 3rd person camera. Combined with the absurd brightness in the elevator’s interior amongst a very noir game, the tell tale “elevator muzak” is playing. If you shoot the overhead speaker, Max utters a smirk filled, 2nd word annunciated “Thank you!“.

Romance, fear, death. In such a confined space clearly only meant for short periods of stay, elevators can force us to view the human condition ever more clearly. Many writers play up the claustrophobia aspect, the lack of control as a metaphor for life taking us where it wills to, or preying on our fear of the unknown when you hear bad things just outside the lift walls.

It also helps players in games get from one floor to another in a controlled manner. Kind of like they do in the real world.

How Does Mine Need to Work?

A game I’m building requires that a character inside a building go up a floor in the building he is in. It’s based on a Brooklyn apartment building that has 13 floors with 12 ft ceilings. The character is on the 12th floor and needs to use the standard elevator. It has a call switch on the outside. You press a button and the elevator will arrive at the floor the button was pressed. You can optionally walk into the elevator and press the up or down buttons for the same effect. The elevator does not move on it’s own, only when a button is pressed. It currently only operates on 2 floors.

Simple, right? Code wise, it was. Here’s the Lua class on Github. The only thing to really note is the goUp, goDown, and tick functions. The goUp and goDown simply set flags so the elevator knows where to go. The tick function handles the actual movement via the game loop. Distance joints, which I’ll cover in a minute, are not powered by motors, so I just increment or decrement the length to move it up or down.

Brief On Box2D

I’ve fully committed to Box2D. That means that a lot of things are handled for me such as gravity, forces, and collisions… basically all the physics code. If you’re not familiar with Box2D, here’s a brief synopsis.

From the site:

Box2D is an open source C++ engine for simulating rigid bodies in 2D. Box2D is developed by Erin Catto…

It’s basically a physics engine for games, although it can be used for other things as well. It’s been ported to a lot of other languages and platforms, such as ActionScript, Dart, and JavaScript.

It’s recently regained popularity because it can ease the creation of Angry Birds like games.

Box2D Constraints

You give up a lot of control when you use Box2D and also have to develop a certain way. For example, all bodies (or things in the world that can bump into each other) default to dynamic. This means they are affected by gravity, which you usually default to 9.8 to simulate the real world.

If you’ve read the Jailbitch tutorials and/or others of it’s ilk, you basically write code like:

thisObject.y += speed;
if(thisObject.y >= theGround.y)
{
	stopFalling();
	thisObject.y = theGround.y;
}

Notice that gravity is applied, by you, on a per object basis. No code, no gravity. No code, no collisions.

This makes building elevators, or “platforms that raise up and down” pretty straightforward. You just increase or decrease the y value of an object. You can detect to see if the character is touching the elevator, and make sure she rises and falls with the platform.

Using Y

In Box2D, all dynamic objects have gravity by default and all the collisions are handled for you. Additionally, how the collisions happen can be tweaked on a per object basis if you like. Things like how heavy something is hitting something soft, the friction, and the bounce. You do not have to write any code for this; it’s just how the world works.

However, making a dynamic object in Box2D rise and fall isn’t as straightforward as you would think. Box2D has a neat feature which allows developers to set a body’s x and y value, and it’ll apply the necessary gravity, forces, and collisions at the next redraw/tick. However, if you want something to move up, you have to treat it JUST like you would in the real world: through force (NSFW).

If I immediately set an apple above my head at a y of 0, it’ll eventually fall on my head. If I want to stay there, I have to keep setting it to 0 using an enter frame/interval/timer… like a juggler. That’s redonk.

Using Forces

Box2D provides, at least in Corona SDK, a variety ways of applying force; as an impulse/jolt as well as force over time. The 2 in particular are body:applyForce and body:applyLinearImpulse.

The applyLinearImpulse is momentary jolt or shove.

The applyForce is like a jet pack; you have to continually apply it via a timer/tick/enter frame to move something.

As you can see, this makes things more complicated. Instead of simply “moving an object by changing it’s y position” you’re now having to compensate for the gravity of t3h erf.

Using Kinematic Bodies

The other simpler option is to utilize kinematic bodies. Instead of dynamic, kinematic allows a body to be affected by forces, like things bumping into it, the functions above, but NOT gravity. Ideal for an elevator, right? In theory yes. I’ll explain why I didn’t end up going this route in a bit. The only drawback here is that kinematic objects don’t “look” like real world objects. They don’t “sway” in the wind, or of something pushes it it just slightly moves. It doesn’t look “real” and defeats the purpose of using Box2D. While following Newton’s Laws of Motion, it looks like an elevator in space vs. on Earth with gravity.

The whole point of kinematic is for the user to drag them via touch or mouse, for cut scene animations where a character can still affect the physical world but you need to move elsewhere… or for things you specifically don’t want to be affected by gravity.

Using Joints

The 3rd option is constructing a realistic elevator so that Box2D can do it’s job: simulate rigid bodies in a 2D space. The mantra I’ve heard repeated by those more experienced in Box2D is keep it simple (or just abandon Box2D and write it yourself… but um… no). Instead of building a real car in Box2D, just build simple geometry (bodies and joints), and let the image of the car itself make the player think it’s real.

An elevator in Box2D terms is basically a box you can walk in and has a cable that attaches the box to a firm ceiling. This means, you create 2 boxes and a joint.

…but what TYPE of joint? And how do you apply it? THAT is what took 4 days.

Version 1: Distance Joint

A distance joint is a line that connects 2 things. You can shorten the line, and like a fishing pole, will reel in the body it’s attached to while it’s dangling below. Perfect for an elevator right?

Bleh.

My first problem was the box was off-kilter. I made it in a C shape so the player could walk in with a wall to the right, and a ceiling to connect the joint too. Being a non-square, however, this made the elevator’s weight distribution un even. It would immediately rotate even if nothing was on it because the weight of the right wall was heavier then the left side with none.

While increasing the friction the elevator had the consistency of velcro, thus ensuring our character wouldn’t fall out, if the player DID walk left, they’d plummet to their death in the elevator shaft, even if you made the shaft just wide enough to fix the box. While you could shrink the shaft and increase Box2D’s sensitivity, it’d sometimes get stuck.

While I could set isFixedRotation to ensure the elevator no longer rotated, and make the distance joint a little more like a rubber band via its frequency and dampingRatio properties, it still didn’t look as cool.

Version 2: Multiple Distance Joints

Most modern elevators nowadays have a variety of attachments beyond a single cable. Magnetic locks with tracks… and usually more than 1 cable. So, I did the same thing and added 2 more cables to compensate.

At first, I was like eureka. But then I hit problems, and was like, no wai!

For some reason, if I changed the length of the first joint to reel in the elevator, AND set the other 2 joints to have the same length… sometimes it didn’t work. Yes, the values would change, but the elevator would just remain stuck. !? I don’t know if this was my code, the Corona simulator, or… Box2D.

At any rate, our intrepid heros can’t run from zombies if an elevator “sometimes works”. While a great plot device, it’s only good if I can control said plot device.

Version 3: Piston Joint

Piston joints are just that: a piston. You can control and constrain their linear movement so moves just like a normal hydraulic piston on carnival rides, etc. Fine, I’ll lose the elevator swaying/dangling behavior, but with piston constraints, it should make the coding easier, right?

Wrong.

Piston constraints, and Box2D in general in Corona totally breaks down when you introduce groups. In HTML, this is div’s within div’s. In Flash, this is Sprites within Sprites. In Corona, Groups are basically the same thing. When creating a game level, you can put the whole thing in a group, and just move the group to move your level.

Box2D, however, doesn’t like this at all. It assumes all bodies are in a same group. While you CAN put things in different groups, things get visually very strange.

Here is some boxes on top of each other.

Now, here’s where I move the group the 2nd box is in. You’ll see boxes pile on top of one another even though visually they are floating in space.

This change of global space also affects calls to the pistons setLimit function. Given Lua in Corona doesn’t have a lot of built in methods to affect coordinate space on the fly, you’re pretty much forced to keep things in a single group to keep it simple. While fine, you still have to CONTINUALLY update the coordinate space… and that doesn’t work if you a lot of classes affecting this as well as the fact that joints can move.

Version 4: Kinematic Floor

I knew I could knock this out in 20 minutes in Flash. So… I coded it like I could code it in Flash: Make a box, decrease its y to move it up, increase its y to move it down. In simple tests, this worked great. Again, I lost the realism factor (the whole point of using Box2D beyond saving a lot of code writing for collisions), but whatever, you MUST git-r-done and move on. Analysis paralysis == bad news bears. Kill t3h scope creep, man.

…however, once I applied collision filters, this fell on it’s face. The same exact code works just fine using a dynamic body, but kinematic? NOoooooOOOOOooo….

Debugging collision filters are impossible; they either work, or they don’t, and your only 2 options are to check your math again or give up. The most frustrating part is that collision filters are the main reason I like using Box2D vs. my own code. When they break (or appear to… no way my math is wrong… ever… specially when I’m changing the body type… right?), they erode to whole purpose of using Box2D in the first place.

I have a rule: if you can’t solve something in an hour, it’s time to move on. Whether that’s another problem/bug/feature, or solution in this case, so be it. So that’s what I did. My #’s checked out, but even in the isolation tests I did, the kinematic box would NOT collide with my character and no collision events were dispatched.

Things got REALLY weird when I created complex bodies using filters for each part; you can see the kruft multiple body code in the elevator class on github. Totally unrelated geometry would act very strange. I’m pretty positive it’s a bug in Ansca’s implementation as I suspect most are merely creating a single polygonal body for complex shapes vs. multiple (like I was for my floor, wall, and elevator ceiling).

Maybe if I spent another hour with a lot of sleep and no kids around I could figure it out, but the rule is there for a reason.

Solution

So I went back to Version 1 and just turned isFixedRotation to true. I also found that sometimes the elevator’s distance joint would get stuck at the top or bottom and even with a high maxMotorForce, the joint was simply incapable of dislodging the elevator. Worse, the values WOULD continue to change and I’d receive no code errors, and visually nothing would change. Simply applying some light applyLinearImpulse (a quick jolt) up or down fixed it every time.

Now, when my character passes by an elevator switch or touches the elevator, the GUI appears for him to go up or down. He’d sometimes fall through the 20 pixel floor, so I increased the Box2D position sensitivity from 8 to 10, slowed down the elevator a tad, and this seemed to fix it. Increasing the Elevator floor’s height + density seemed to help as well.

Conclusions

As you can see, what I thought would be simple was made harder by my not knowing the quirks of Box2D, nor having built an Elevator with floor collisions before. That said, Box2D is really fun, and I can see how the more I learn about it, the more time it saves me for certain types of things… like my Die Hard fire hose.

Basically, the things I need to keep in mind and learned:

1. Keep all Box2D objects in Corona in a single Display Object; inject child dependencies if a class needs to create composited objects
2. If a joint has a length & motor, but things get stuck, try increasing the body’s density, reduce nearby body’s friction, and applying a tincy bit of impulse to “git er unstuck””.
3. Create visual test harnesses; a place where you can test your components in isolation. It’s amazing how frustrating debugging reference points are in Corona SDK as a Flash Developer. In Flash, everything is aligned top left unless you move stuff manually. You can’t even edit the registration point. In Corona, my convention is to default everything to top left to emulate how Flash works, and thus make my visual alignment coding appear cleaner… except for groups. Being able to test the elevator alone, and then in game, allowed me to quickly diagnose problems.
4. To heck with you piston joint. You’re awesome, but until Corona makes it easier for you to adjust being in a different global coordinate space, I’m sticking with distance and touch joints.
5. I really wish I knew why my kinematic bodies aren’t working with my collision filters. In isolated tests, it works just fine.

I hope this helps you in your Box2D adventures.

I’m in a hurry.

I don’t want a history lesson.

Get comfortable, this is long.

Preface

I had the pleasure of attending a 5 day fast track for Sencha’s Ext JS 4 in New York last week with my partners from Web App Solution. Of the 13 people that attended, 7 were Flex Developers working on Enterprise Applications. Yakov and Victor from Farata Systems were there and Universal Mind had John Yanarella representing.

While the technical future of Flex is extremely bright and exciting, the business and marketing future of it is over. Adobe, with the community’s help, has ensured that it is in the best hands with the Apache foundation. With a multitude of capable, brilliant, and diverse contributors to the open source project, there are definitely a lot of great things on the horizon for the next Flex SDK.

Adobe has also ensured, intentionally and unintentionally, it has a limited future which will ensure Flex is no longer invested in any technical capacity from small, medium, and large businesses unless absolutely essential, or under the radar. The mobile revolution has happened, and while Flash had the opportunity to play a pivotal role, Macromedia, and later Adobe have failed to take the initiative and successfully execute on it.

While there is still a significant business & technical advantage from a design, development, and workflow perspective in the Adobe AIR stack for developing & deploying on Android and iOS, Adobe has made it a clear that they perceive it is an opportunity they are not interested in pursuing on the developer front. This is a huge topic mostly discussed from a technical perspective in the community, and only a select few blog articles got it right from a business one… and even then, those missed it from a technical perspective.

Bottom line, Flex Developers are between a rock and a hard place in the industry right now.

The type of work they do for desktop web applications is different from what Flash Developers do, mainly in team size, project scope, and client base. It’s not as simple as “just use jQuery, and move on”. While the tools & runtime we use are technically sound, even clients who have a technical understanding of the marketplace & toolset will not invest in a perceived abandoned platform. Server-side offerings do not offer the rich client experiences we’re used to creating. The plethora of frameworks in the web market, both open source and proprietary, do not offer the same tooling, performance, and expedient workflow we have on the Flash Platform.

… except Ext JS. Sort of.

As stated by John Yanarella at the training last week:

“Out of all of the web based frameworks out there, Sencha’s Ext JS is the best safety net there is for Flex Developers.”

It has the necessary component framework, a growing company with enough resources & funding at a specific size behind it, with a competent web & mobile technical story that has the potential for critical mass within an Enterprise context.

Thus, given my company’s need to offer my clients an HTML5 alternative to Flex, Ext JS seems to fit the bill. Considering time is at a premium nowadays, the fast track training Sencha offers seemed like a great opportunity. And I love New York so any excuse to go there is great.

Given the whirlwind that was the past 3 months, as well as reflecting upon what I learned from the training, I thought it prudent to write down my thoughts here. There are a lot of Flex Developers who are looking for direction, so I’m hoping this is one for them to follow.

Background

As a Flex Developer faced with a forced career change driven by forces beyond my control based on the infamous events of November 9th, 2011, I had 2 choices. Do what I always did back in my 20′s and dive headlong into the unknown, or let the pioneers get the arrows in their backs as it were. Now that I run a business and am a father of 2 children, I don’t have the free time I used to back being single and W2. Flash & Flex, even on death march projects, allowed me to get up in the morning fueled by the passion of using my favorite technology, and attack problems full bore.

Surveying the landscape, things were pretty grim… even for someone positive like me.

While the iOS stack had a clear advantage with both a growing developer base, mature tooling & framework, and customers who actually paid for content, none of my large clients actually had initiatives around iOS work. Enterprise applications, known for their copious amounts of input needs from the user, do not translate well to the small screen size and lack of keyboards on smart phones and tablets.

As such, while the uptick in service based software shops servicing the needs of clients who need iOS applications has increased, the scope of said projects is not as large and often is more consumer based. The latter is troubling as it’s hard enough utilizing ActionScript 1 in a deadline driven environment that Design Agencies employ; dodging technical debt for as long as possible in Objective C, even with superior tooling is not appealing when longer and more profitable projects exist with the opportunity to have competent peers beside me vs. being a lone wolf. Learning a lower-level toolset for shorter engagements that pay less makes it a non-starter.

Android, while being easier to deploy to as well as more easily iterating with your customer base, is notorious for it’s fragmentation, both programming and design. It’s hard enough for designers to lessen the overall experience via responsive web design to compensate for a variety of screen sizes. You now have developers who cannot turn as fast as designers can and thus have to spend more time on a platform that doesn’t generate nearly as much, if any, revenue from consumers compared to iOS. This doesn’t even broach the business topic of how do you lessen the maintenance cost on your customer when they have to hire both an iOS and Android developer, or some awesomely expensive hybrid who can do both, to handle both platforms. The other bizarre thing about Android is that it is taking the PC approach to competing on price with iOS and continually increasing market share, yet doesn’t seem to be generating the service revenue compared to iOS (ie people who own Androids don’t buy apps as much as those on iOS). Same problem as iOS: Invest in a lower-level technology with massive fragmentation for shorter projects that pay less money… again, non-starter.

The multi-development & deployment problem was what the web was supposed to solve. While native development does give you significant user experience & tooling advantages,   it seems wrong to invest my career for applications that my current client base do not target. Every single large client I’ve had treats mobile as either a fun skunkworks project, experiment with no perceived ROI, or as a necessity with the minimal amount of money devoted to it, sometimes taken from another project’s budget ad hominem.

Both don’t bode well for a future career from a consulting perspective in mobile.

While consumers bringing their smartphones, tablets, and technology of choice to work has had a huge impact on large companies, most Enterprise customers still target the desktop web as the main avenue of both B2C & B2B based revenue. While many have made huge and wonderful strides in providing value for their mobile websites, it’s clear many either provide a lot less functionality than their desktop counterparts, or strain at even that.

With large media companies it’s a little different. Studies have shown that tablets are better at allowing consumers to consume content whereas desktops/laptops are better for producing content. While “duh”, having studies like this made and corroborating themselves verifies what we already know. They have a vested interest in increasing consumers ability to purchase and access that content in a controlled manner.

The problem here is that the budgets for those projects doesn’t increase with the needed targets.

For example, while working on HBOGO.com, the initial release of the iPhone app did not allow you to watch videos. The Flex app we did at the time was the only way to view videos. Overtime, it was prudent to support both iPhone and iPad versions. A lot of companies, however, don’t magically get 2 budgets, 1 for the web and 1 for the iPad just because the iPad is so cool. While the GUI’s needed for iPhone and iPad are simpler, you still want to create the same user experience for the customer, especially if they are using all 3 to access their content. The Iron Triangle verifies that something must give, and thus, the scope/functionality/user experience often does.

Investing in a mobile technology where consumers pay less, and larger businesses devote less capital to the same projects doesn’t bode well for investing the smaller portion. If I were back in 2002 still doing design agency work, I’d totally be an iOS developer right now. As a consultant who does more than build software, and who doesn’t see his clients changing their plans, I see no prudent reason to go the iOS/Android device route. …… beyond them being really fun (another story).

Appcelerator, Corona SDK, Sencha Touch, and PhoneGap, technologies which allow you to deploy to multiple mobile platforms with mostly the same code base and skillset, can be lumped in the same bag as native with the above perspective.

Why invest in technologies that offer less money and my clientelle isn’t using?

On the flip side, the HTML5 moniker has taken on a life of it’s own. While not necessarily  representing the full JavaScript, HTML, CSS, and associated libraries & frameworks stack, a lot of clients big and small have been sold the bill of goods that it solves the same multi-platform, multi-device problem that the Flex/Appcelerator/Corona/Touch/PhoneGap attempt to solve: single code base, single developer toolset, and multiple targets within their same organization.

I won’t get into the phenomenon that is HTML5. All you need to know of the context of this post is that a lot clients either suddenly think HTML, CSS, and JavaScript can provide the same experiences we can do in Flex for the same price using the same teams… or they know it’s false, but either their Board of Directors or Business Customers force them to use the tech anyway for political reasons that have nothing to do with engineering ones.

The Challenges of the New Platform

The issue, however, is that it’s not that straightforward. There are 5 main issues that the HTML/JS/CSS stack (which I’ll here after refer to as the HTML stack), have when coming from the Flex platform in a consulting context (there are more, but these are the ones that matter with regards to the context of this post). They are:

1. DOM as a dynamic display technology
2. JavaScript lacking mature language constructs
3. There are larger problems for multiple Browser Vendors to solve
4. the industry’s libraries & frameworks are young from an Enterprise context (yet not!?)
5. The Video Codec Issues
6. Tooling

The Document Object Model

The HTML DOM wasn’t made to do the type of applications we do in Flash, more specifically Flex. Flash Player was developed specifically around cross platform visual fidelity with a web bent. More importantly, though, it was built around redraw. This perspective of “frames per second”, and continual improvements around the underlying timing of redraw, building upon what a platform does well, and then abandoning a “new” 5 year old display technology to support a GPU future.

Browser vendors are doing some pretty amazing things with the DOM, from hardware accelerating the display, continuous redraw optimizations, all the while doing their best to follow ever evolving standards amongst our changing technological landscape. …based on a tech designed and created to show text. That’s awesome.

Sadly, the redraw performance isn’t awesome. While a lot of Flash and Flex developers never really got the purpose and value of invalidation in Flash Player, even with a technology created for displaying and animating composited graphics, as your application scales, it’s easy to create a system that redraws too much. Invalidation helps mitigate (usually solve) that problem for 90% of the use cases in applications. In a nutshell, it means you can set properties on your visual components as much as you want to in a single thread and ensure it only actually redraws the screen 1 time.

For browser DOM’s, this is exemplified even with the drastic CSS improvements. The philosophy is to never redraw the DOM if you have to, and if you do, do so on the smallest section possible. Browser toolkits such as Ext JS,  Dojo, and some others do this very well. While they have the advantage of browser CSS to help mitigate some of the redraw performance drags, compensating for both past, current, and future browser incompatibility in an easy to use & scalable API shows just how brilliant some of those browser developers really are. Based on what I’ve continued to read, this seems to be getting better although not as fast, nor as widely deployed, as Flash Player did which means the problem isn’t going away soon and thus the need for x-browser, x-platform invalidation continues to be important. Thus, Ext JS’ commitment to efficiently redraw the DOM is appealing.

One thing John brought up at the Ext JS Fast Track training last week was the lack of positive sales around hybrid solutions: using Flash Player and the HTML stack together. If you need to play video or animate/show a lot of composited objects, combining the 2 technologies together.

Hulu and YouTube proved it can work with video and movie subscriptions. Grooveshark proved it can work with music (and video, heh). Google Finance and Analytics proved it can be done with charting.  Balsamiq proved it can be done with desktop designer tooling.

Bottom line, things are pretty bad but getting better slowly, and there are mitigation paths that can utilize Flash Player in the interim. Canvas is finally getting hardware acceleration across platforms also making it another another tool to help.

This doesn’t seem to be getting much play, but considering we’re all supposed to magically know a multitude of platforms, languages, and yet have multi-talented sub-contractors that meet that criteria and deliver the projects in the same time frame clearly means that option is also on the magical table.

JavaScript in the Enterprise

Surprisingly for me, JavaScript the language has dominated the discussion the past 3 months in where I’ve lurked in social media circles amongst the Flash & Flex Developers. For those of us who grew up in the industry and our careers with Flash, it came quite as a shock for the industry to fall back in love with JavaScript.

While it made sense for server-side developers to reduce their code size as more logic goes to the client and their servers get faster and cheaper, we’re still a little confused as why the industry isn’t salivating for a statically typed Harmony (resurrected, non-elven remains of ECMA 4).

Most of us Flash turned Flex developers grew up using ActionScript 1. For those who don’t know, this is near exactly the same as JavaScript/ECMA 3. The difference is instead of doing an innerHTML on a div tag, you just call methods/properties on a Sprite. As our projects grew in scope and # of developers, the language started to show its scalability problems. You’d run into more and more null pointers from misspellings, even with better tooling to help provide code hinting. You’d also get errors, but no clue where they came from since there was no common way of handling errors in ActionScript 1.

Furthermore, if you didn’t provide source, the developer was screwed because there was no way to easily introspect API’s unless you read the documentation or source; things like Aptana didn’t exist back then. This also implied, however, that even though there were things that weren’t public, developers would still call them anyway if they didn’t understand your name spacing convention (such as using an underscore (_) prefix for private variables and method names). While Lua and other languages have tricks using closures to emulate access modifiers, this never caught on in the early Flash world. The path for RIA’s at the time was larger projects with larger teams, and thus they adopted a more Java esque method of language development while still trying to match ECMA 4 as best they could.

Finally, loose typing/variants also made it harder (in theory… which has been proven wrong by Google’s V8) for Macromedia/Adobe compiler engineers to make the language run faster than JavaScript.

Thus, the natural evolution was for a strongly-typed language with proper packages and namespaces (or access modifiers like public/private) to help us address the larger applications we were developing. It did. It was, and is, awesome.

…for some. About the same time as I started getting more consulting, I also started noticing a larger awareness to the different types of businesses using Flash and Flex. In the Design Agency world, the majority of the work was 2 months to 2 days, and in between. The teams were often 1 developer; if there were more devs, they both weren’t on the project the whole 2 man months. They hardly ever payed their technical debt (unless you were a consultant with them). You started to see a strange love/hate relationship for a lot of these devs. In some cases ActionScript 3 helped these developers, who were on a deadline and often never paid their technical debt, manage said technical debt with AS3′s improved strong-typing and runtime exceptions with the ability to use loosely typed constructs when needed or in unknown territory.

On the flip side in the Flex world for larger companies that gave a flip about design, and more about feature count, the language constructs were barely enough. Constantly we’d get barraged from developers in more low-level languages like Java, C#, and C++ about why is AS3 is missing this or that (true Singletons, true Abstract classes, threads, blah blah blah), and how do you emulate such features.

It was very clear there was a split was growing. I could elaborate on all the niceties, but suffice to say, the Flash Devs didn’t need all the language features & strict complexity rules whereas the Flex Devs couldn’t get enough. Again, a simplification on the nuances of  a thriving, diverse, and changing community, but accurate.

Most of the Flash devs have had little trouble beyond the first 2 hellish weeks of getting immersed back in the DOM, CSS, and JavaScript variant world… only with better tooling and libraries + choices than they had 7 years ago when they 1st abandoned it for more creative pastures.

The Flex devs, however, haven’t seemed to budge en-masse. We have people telling us we don’t need those things in a language, yet when we ask them to show us a large Enterprise Application that utilizes JavaScript, they either are using an intermediary language to compile to it (Java in Google GWT or Oracle ADF, C#/VB in .NET, or CoffeeScript), or just can’t deliver the goods. Same excuse us Flex developers gave the Slashdotters around “show me a cool app built with Flex”: “It’s behind the firewall”.

There’s also a lot of misunderstanding about simple things like OOP. Using Object.prototype for “emulated” classes is what I did for 4 years… I have no major qualms about going back. Ext JS in version 4 abandoned this practice as well. While I grew up with Object.prototype insanity and learned to love it, the debate vs. clsoure/function inheritance vs. Object.prototype rages still. If you care, Keith Peter’s has a good article on the nuances of it, or you can read what I did to learn ActionScript, Robin Debreuil’s book.  There are others, but the point here is we Flex Devs expect classes not just for encapsulation, but create organized code with a common workflow to share code amongst many developers. JavaScript wasn’t made for classes, but using Object.prototype or the closure method in a declarative manner works just fine (like ImpactJS or Ext JS does).

While I like Keith’s take and respect his opinion, not having classes and encapsulation won’t work in large teams, often setup to fail at the outset by politics, nor all being Keith Peter’s caliber (me included). The other option to “not working” of course (Goonies never say die), which I’ll cover later, is that it’ll just take longer and/or we’ll accomplish less. The rules of Mythical Man Month, Technical Debt, etc. still apply regardless of  language/platform. It’s worse in Consulting when coding software, extremely hard in it’s own right, is the least of your problems.

Compiling to JavaScript

In fact, some are so staunchly opposed to utilizing JavaScript as a serious language to build Enterprise Applications, that they’ve investigated alternatives to compile TO JavaScript using an intermediary language. This has long been the process for server-side platforms with a variety of templating engines out there that make it a hybrid process of injecting data into your HTML. The problem is, they either have a server-side bias, or aren’t ready for prime time in a large Enterprise Scale. Remember, you can’t just be good; you have to have money, a company of decent size behind the tech, funding, and have reached critical mass for your product.

While the reaction was very positive around the work Adobe was doing with FalconJS for those in attendance at the FlexSummit, it was very clear to me it was a science experiment with no hope in providing the needed migration path for Enterprises. The amount of work, resources, and time needed to bring it to Oracle ADF, Google GWT, or .NET caliber is clearly something Adobe didn’t seem willing to devote. That, combined with their clear direction of no longer focusing on developers.

Google Dart seemed like a light at the end of the tunnel. It, too, however, seems to be a science experiment of Google scale. It has all the hallmarks for a bright new project:

• Google is behind the engineering effort
• has a strongly-typed language that, like CoffeeScript, not only sets out to improve upon JavaScript’s problems, but provide mature language constructs you need in larger applications within the language itself (unlike CoffeeScript’s/Google Closure’s/TypedJS annotations
• has 2 compilers: 1 a Flex Developer would use (dartc) and 1 a Flash Developer would use (frog).
• is based on Eclipse (good and bad, let’s just pretend the good parts here)
• brought on Alex Russell of Dojo fame to take a fresh look at the DOM event API
• Cliff Hall of PureMVC fame (multi-language framework used a lot in Flex) made a first version port for Dart.

…yet doesn’t seem a priority since while a web based technology, it still hasn’t reached critical mass, nor does it show any signs of doing so yet. I’m more concerned about the amount of resources I’m seeing being devoted; maybe there are more, and like all things Google, they just don’t advertise it. I really wish I could just parachute drop a Porter & Bogusky contingent in Google someday…

So far, the only contender is CoffeeScript. Here is a video and slide presentation to check out on if you have time. However, Ext JS uses declarative JSON + JavaScript. I have not seen any signs to see if Sencha is interested in porting Ext JS to Dart. Just say it… “Ext Dart” (pronounced “X Dart”); sounds like a dope dubstep ninja weapon, right? You’d certainly get better jokes from non-tech savvy people at parties when someone hears you’re a “Flash Developer”.

The other problem is that you need everyone in your organization coding the same language. In product companies, it is TOTALLY fine to have a technology du jour with Ruby running your builds, PHP running your render farm, and Java running your back-end, but for development purposes on 1 particular platform, you standardize on a language and usually have company ordained standards on it. This is for maintenance costs, and hiring standards, and a variety of other reasons that aren’t just around cost.

If anyone has CoffeeScript running atop Ext JS, let me know how you got your build process setup.

Browser Issues

Notice I’m specifically talking not just about incompatibilities between browsers, but missing features. For example, just 3 days ago Google added weak collection keys into V8 (their JavaScript engine for Chrome). Developing data models with string based keys can be extremely hard for complex model logic. Having a Dictionary (or Object) that can use something other than a String as a key is extremely useful… until you have to remember to kill the key when the object is destroyed. Enter weak keys and queue the “woo hoo!”.

One of the great things about Google’s approach with Chrome is not only is it the fastest updating piece of software that matters in history (faster than Flash Player), but they iterate quickly on features. While we’d wait 18 months for AS2 and then 24+ months for AS3, and in between get language updates… JavaScript is getting that piece by piece. It’s a wonderful thing, but also brings up a larger issue.

What do you today?

You have to do SOMETHING. Given each browser implements different features different ways, and JavaScript’s functional + prototype nature, you can easily (*ahem*) abstract away these problems into a useable API that hides the differences. Certain low level things like maps can be tricky, however, and come at the cost of performance, library maintenance costs, etc.

Fine, no big deal, you just move forward using an Object hash approach. We’ll just wait a couple of projects before maps + Dictionaries catch on in all browsers.

…but what about Web Worker (also known as threads for web developers)? It’s only supported in IE9 while Firefox/Safari/Chrome are good to go. While you can emulate via green threading, this is a FAR cry from emulating the lack of getter/setters in older versions of IE, or even emulating Canvas for Internet Explorer 8 via a VML wrapper. Those work. Green threading is not the same thing as a separate process that ensures it doesn’t lock up your GUI thread. The lack of full browser support will prevent some features from actually working and there aren’t any fall backs that can be utilized.

For some charting/plotting applications, these are essential features: no web worker, no application. Now in Flash, you’d just plan around upcoming features, both those you voted to be actual features, or those you knew were coming down the pipe. One tactic of design agencies, and Macromedia/Adobe in general was to have them launch a high profile app/site/widget to showcase these features upon launch.

If you are to inspect all upcoming features in browsers, not just JavaScript, but CSS, hardware & display related, etc. it really slows down when we expect certain experiences in our applications. While Macromedia/Adobe would drag their feature on certain things like right click and threads, the process was generally the same:

1. identify a problem you had on projects
2. bring it to your peers & Macromedia’s attention
3. rally support through email list/blog/twitter posts/talking to company reps at conferences
4. make writing & validating the use case easy
5. get community votes when desperate
6. await new feature(s) in upcoming Flash Player, beta test if lucky

While this process for Chrome seems EXTREMELY quicker, for the rest of the browser market as a whole, including smart phones and tablets, not so.

My college professor said, “There will always be work to be done.” Unless we use hybrid implementations, clients will just have to live with the slower pace of innovation if they want broad adoption. I guess I’ll live, I’m just curious where my excitement will come from if the toys remain the same, or only 1 browser implements a new feature I can’t easily abstract. That, and it seems a lot of innovative projects will either start to be browser specific feature branches, or just wait for broader adoption amongst modern browsers. It’s sad. And frustrating for both my clients and my team.

Wild West: Libraries and Frameworks

What’s fascinating to me about the JavaScript landscape is how much longer it has been around with a lot more people and yet the landscape with regards to frameworks and libraries is still in major flux on a few fronts.

In the Flash World, there were 2 frameworks that people would use, if ever: ARP and… just kidding, 1. While the tweening libraries got a little out of control, and the component frameworks became pretty niche, overall there was a library for everything, sometimes with an alternative if you didn’t like one particular implementation. A lot were ported from other languages, like Java and C/C++.

Frameworks didn’t really start to rear their head until Flash RIA applications started to become mainstream, and Flex was born. The Java developers coming over didn’t see any common MVC/MVP/MVVM frameworks that allowed them to manage copious amounts of code on the client side vs. middle tier. Cairngorm, PureMVC, Swiz, Parsely, and Robotlegs were born. Cairngorm is still around, but Swiz/Parsely get most of the play in the Flex world with PureMVC and Robotlegs hitting both Flash and Flex.

With exclusion of Cairngorm, all 3 have a lot in common. While everyone somehow manages to make their own implementation, learning one helps in learning another. Additionally, most never made it past a version 2, and even then most of the changes weren’t a dramatic workflow change.

JavaScript is nuts on the framework front. While the libraries themselves are great in that once you get over the burden of reviewing 5 of the same thing, the frameworks on the other hand are all over the map. Some are just query helpers with binding thrown in. Some are actually a certain group of existing libraries with a defined way of working with them, and thus named a framework… even though no code is actually associated with the framework itself. Worse is the versions can sometimes be drastically different. If few in #, this wouldn’t be a problem.

Worse, there are 100 times more people in the JavaScript community with their own opinions on what MVC means. What MVP means. What MVVM means. What the differences are between the 3. Their own descriptions for what the differences are between Passive View, Supervising Controller, and Presentation Model… and sometimes pretending (innocently) that the only thing that exists is MVC, period.

The other thing that is clear is that there are a variety of applications (not progressively enhanced websites) that are done with JavaScript, and thus each framework targets that particular version. For example, PureMVC and Robotlegs have Mediators which make it easier for the View (what you actually see) to change a lot during the project, yet still keep its ability to get it’s data without race conditions, and have it produce the necessary  user gestures through Events. Swiz on the other hand, had a Presentation Model injected into the View, and you had a nice API the View could call/get it’s data from. A lot of Swiz guys I talked to NEVER used the states features in Flex 4… nor saw the point. GUI changes didn’t affect PM API, they weren’t complex enough that you had to use mock interfaces for the PM’s to test View’s in isolation… so for this group, it worked well. In retrospect, this was a nice, natural evolution that happened.

I’m seeing the same thing in JavaScript, but it’s massively larger, and a lot of times you’re cherry picking either a specific feature of a specific library to be used with other ones, and you’re settling on a version. The last one is worrisome from a consulting perspective. When dealing with clients, Technical Architect/Directors want to ensure whatever framework you settle upon has good online documentation, you can hire people who know it, and it has a future. Using the same metrics I use to do so for frameworks in the Flash/Flex world is a little harder; just because its someones side science project doesn’t mean it’s not immensely useful, nor easy to learn.

The other challenge from a consulting perspective is leaving clients with work that they can support… or my firm would even want to get involved in. For example, there are copious reasons Ext JS 4 moved away from Object.prototype modifications. If you adopt a Ext JS 3 project from a client, you now adopt those problems even if you didn’t write it. With Flex projects, you eventually learned what problems there were from porting a Flex 2 app to 3, or what parts of mx were supported in Flex 4, and could find blog posts covering these challenges. With JavaScript, I’m finding a hard time finding commonality between the stacks that different companies are using.

I. Do. Not. Want… to be a non-Enterprise TA right now. Holy fish. Like, it’s 2008: “Yes, we’re using YUI version 2, it’s rad, has a large company backing it, and we can hire for it.” It’s 2012, and you’re either a jQuery, Sencha, or Dojo kid… oy vey.

So far, with the exception of Ext JS’s MVC implementation, it seems to have all of that solved in version 4 as best it can with the ability to have your own libraries added in with your own modified build process over theirs as well if you see fit to do so.

Video Codecs

No one has solved the codec problem yet. This is where HTML5 video just doesn’t work as advertised. I won’t go too much into this topic as for Enterprises, you’re usually more concerned about VOD, DRM, multicast and live streaming which Flash owns for desktop and HLS owns on iOS. More info about the WTF here and the OMG here.

I only bring it up because it’s just accepted in our world that video and compositing of it + assets is expected in our content, yet now… it’s not. Good designers can make it work, but the point here is from an integration perspective the cost of integrating that content.

Tooling

Lastly is tooling. While Aptana makes an attempt, it seems that Microsoft’s Visual Studio is the only IDE that seems to take web development seriously for those building large applications. Since me and a lot of people I know in Flex development use Mac’s, it seems our only alternative is JetBrain’s WebStorm. Granted, they have other flavors of their IntelliJ IDE,  but without commonly expected functionality like code hinting, intelli-sense, and re-factoring, it’s hard for use to take web development seriously.

There is a huge productivity cost that people seem to forget when you move from Eclipse/IntelliJ with those features to Notepad. When you have multiple developers, that time spent adds up, as does the lack of accuracy on your team’s velocity in solving issues.

In 2004, a lot of Java Developers tried out Flash. They then left. Adobe then created Flex. Some came. When Flex 2 came out, the runtime worked, the language worked, the components worked, but more importantly, the Eclipse IDE at the time… worked. The refactoring was, and still is not the best compared to IntelliJ and Visual Studio, but it’s enough for them to take it seriously. That’s when they came en masse, and peeps like Bruce Eckel were like, “Flex is good stuff”.

Going into an Enterprise and telling developers they are no longer allowed to work on Enterprise software with intelli-sense and code hinting isn’t going to work. Thankfully Ext JS is working with a few tools vendors, and some, like WebStorm, can be customized to improve the hints you get, including on your own code.

At the end of the day, we Flex Developers are concerned about the overall reduced project velocity & resolution prediction that comes with losing static typing (“What and where is the problem and when is it going to be resolved?”). For smaller projects with smaller teams, I guarantee you the point is moot. In doing both, if you CAN do both, dynamic languages are so much faster and fun to develop quickly, and flexibly in. For 18 month, multi-developer teams on code that’s alive for 5 years, it doesn’t.

Introduction

As a business owner, we have to meet the needs of those clients that have adopted the HTML Stack, or feel they need to and cannot be convinced otherwise. In the following article I’ll cover what Ext JS is from a Flex Developer’s perspective, why you should care, and some conclusions on how using it + Flex changes our business for the better.

As I mentioned before, the events of November 9th, 2011 shook the Flash & Flex world to the core, and I had 2 choices: figure out where to go, or let other key members of the community figure it out.

I chose the latter plan: play Skyrim and disappear for 3 months to let it all shake out. If things hadn’t been figured out by key members of the community, I’d just figure them out for myself. It’d also be around the same time as Groundhog Day: If the Flex community saw their shadow, and freaked out, I could come out and help them weather the longer winter.

It’s been 3 months and so far, it seems all the Flash Devs went jQuery/Backbone (or Angular) whilst the Flex Devs are debating between Dojo and Ext JS (pronounced “X Jay S” if you’re French Canadian) (yes, I tried YUI and left just as quickly). After talking with members of the community and doing some of my own research, it seems Ext JS is the right choice for the type of work I do. Like using Flash vs. Flex, it’s clear it may not be the right choice for you. If you’re a Flex Dev doing large, multi-developer projects, I’ll bet it IS the right choice. Read on.

What is Ext JS?

Ext JS is a component framework for building web applications atop JavaScript, HTML, and CSS. It’s made by a company called Sencha who also makes a mobile version called Sencha Touch.

It’s attractive to Flex Developers because:

1. It has components. Lots of components.
2. It has styling and theming features.
3. It has a Model architecture built around data store & server CRUD operations.
4. It has charting components that do not utilize Flash Player (for good and ill)
5. It has awesome item renderers for grids.
6. It has classes for JavaScript with inheritance.
7. Layouts with Containers
8. Standard Focus Manager
9. Core Library with DOM helpers as well as JavaScript language extensions
10. A big company, with investment, and tons of docs who’ve adopted a prescriptive MVC framework.

There’s a lot more, but that is what we Flex devs actually care about.

How’s it work?

Ext JS is a framework; a collection of classes, 3rd party technologies, and a prescribed methodology for building web applications. HTML is used to show the components. CSS is used to style and modify interactivity. JavaScript and declarative JSON is used tomake things appear and work.

JSON, traditionally used by Flash and Flex Developers for data transfer, is actually the main way you code Ext JS, similar to how MXML makes up most of your GUI in Flex. Since JSON itself can be written in a declarative way, this is usually the core of how you define a lot of your application; both GUI and data. This JSON + JavaScript is interpreted to create classes, components, and listen to event handlers. Internally, the Ext library will redraw the GUI as necessary, whether that’s modifying CSS styles on the DOM objects that visually represent them, or swapping out innerHTML wholesale.

You include a javascript library on your HTML page, code some boostrap JavaScript/JSON on the page, and voila, you have an Ext JS app. It can be as small as using 1 or 2 classes from the Ext JS library, to a full blown application that uses their components, their model classes, their MVC, their default module loader, their implied CSS abstraction layer, and their build system… up to you how much you want to use.

Prescription & 3rd Party Integration

Sencha has to walk a fine line with Ext. From an Enterprise stand point, they have to lay the ground work for a prescriptive way to utilize their technology. You need to make a framework easy to work with, have good documentation, and not be too flexible that developers can all go their own way. This allows larger organizations to hire/cross-train easier, have those developers actually be successful with minimal investment, and basically ensure any skillset “knows how to build an app”.

It’s a sales pitch, yes, but it does need to try to work like that. In reality it never does, hence consultants like me existing, but that’s not the point. You don’t want many different ways to skin a cat from an architecture perspective. It’s fine if you cuddle your brackets, use TextMate to code, and prefer Mercury over Git… it’s not fine if you prefer MVVM over MVC, want to be intimate with the DOM, and the other developers in another part of the company on a companion product don’t. Unlike Design Agencies, Enterprise companies do have to pay their Technical Debt. In fact, I’d argue they pay more than they should, compensating in a bizarre karma like way for Design Agencies who get away it. You want to mitigate this problem by making something easy to use, easy to hack, but prescriptive enough that if you don’t know any better, you won’t cause undue costs down the line in the way your team built something.

JavaScript, currently, is all about libraries, more so than Flex and Flash was for a variety of reasons. Namely abstracting browser differences, abstraction over new features with fallbacks, and the sheer number of developers from a variety of backgrounds. Each company is going to use different libraries merely based on the number of library choices out there. The same goes for internal projects. A lot of companies have their own internal libraries utilized on products for their business, whether B2C or B2B. You need to be able to integrate with these in your native language.

Sencha needs to support prescribed direction in how you build web applications with the ability to support 3rd party libraries.

There are a few things they’ve done to facilitate this. Here are a few that matter.

First, they’ve abandoned Object.prototype as of Ext JS 4. If you’re a Flash Developer, you’ll remember having your language suddenly act different because some developer or 3rd party library overwrote a core property/method in Flash, and suddenly your entire application worked differently… or it only did it sometimes because of later loaded class. Unless you knew what you were doing, these were nearly impossible to debug since the documentation stated a method did one thing, but clearly during testing did another. Even if you did know what you were doing, there aren’t many language, nor tooling facilities to help you debug this inheritance mixin whackness.

Considering JavaScript library inclusion amounts are significantly higher than say Flash or Flex, the risk of collisions are higher. On multi-person teams across an organization, with already strained communication this can be horrible to try to resolve.

Also, from a module perspective (those of you using PureMVC multi-core or many Robotlegs Contexts will get this), you need to sometimes run may applications in parallel on the same page. Not having to worry about stomping on someone’s method/properties is a nice feeling to have.

Second, they’ve commited to bootstrap.js their own bootstrap.js from a library/module perspective which supports both debug and production mode. With the build process, this allows you to develop with new classes on the fly, and in production mode actually have 1 file for you’re entire application. This is for speed & ease of development purposes.

We take this for granted in Flash/Flex where all our classes are included, and zlib compressed in a minified bytecode format into a single SWF file.

Remember, Day 1 of Flash Development had 1 SWF. As a binary format, everything you needed was right there for you, ready when you started. The HTML stack was different. Everything is a bunch of files all loaded at different times from different places. As a coder, you know this is a nightmare to figure out. From a networking perspective, initiating a lot of HTTP requests to the server and downloading them in parallel is also slower. So JavaScript has a habit of allowing you through build processes to put all of your classes into 1 massive file to help alleviate the race condition pain as well as improve performance, compression, etc. However, during development or when building modules/lazy loading, you still need the option to load on the fly.

Third, their MVC is prescriptive. I won’t get into too much detail because as a Swiz/Robotlegs aficionado and PassiveView h8tr, I already want to use my own or Angular… anything but theirs. That doesn’t matter, though, the point here is that they’ve provided the built in mechnisms, it works great with the framework, and they have documentation on it as such. This ensures organizations take them more seriously, have a chance to at least be forced into some known structure, and you as a consultant have a chance of understanding how they did things if you come onto an existing project (assuming they didn’t just use Ext JS for it’s query syntax and get all intimate with the DOM… which you can do if you wish).

That’s Great, Show Code

Here is some Hello World code:

<html>
	<head>
		<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
		<title>Ext JS Hello World</title>
		<link rel="stylesheet" type="text/css" href="/extjs4/resources/css/ext-all.css" />
		<script type="text/javascript" src="/extjs4/bootstrap.js"></script>
		<script type='text/javascript'>
			Ext.onReady(function()
			{
				Ext.Msg.alert('Welcome to Ext JS 4', 'Hello World!');
			});
		</script>	
	</head>
	<body>
	</body>
</html>

As you can see right off the bat, closures are the word of the day. Ext JS is full of either closures (anonymous functions or variables that equal functions) and/or JSON. You’ll intermingle both a lot.

Components

Components are the main reason most Flex Developers dig Ext. Not only are there are a lot of components, but some of the ones they have are better than their Flex equivalents. And this, folks, is one of the reasons why software developers often tell you to try another language or platform, at least 1 per year. You’ll see what the competition has, and YOU’LL GET PISSED OFF!

Why didn’t Adobe build this level of components for Flex? Why does Django, Rails, Coccoa, and countless other frameworks have a Model with optional Facade on top to support offline storage with online CRUD syncing out of the box… and Flex does not? How come our DataGrid didn’t come with built-in paging?

Enough ranting, check them out yourself (scroll down, check out list on bottom left) while I cover a few here.

T3h Über Grid

You’ll notice right off the bat their DataGrid is pretty bad ass. Three things I want to point out about it.

Notice they have paging built-in. This is, in part, to their Model/Proxy/Store setup. In Flex, you had to code this yourself (don’t get me started on Blaze DS/LiveCycle).

Notice their filtering actually has a GUI with a drop down menu to customize (and which YOU can customize) on how to filter the data. No server trip needed. Hot.

And finally… “row editing”. Yes, an entire row can be modified. You can also allow a row to be edited, but only allow certain columns in that row to be edited, as well as customize this renderer. Awesome, right?

Charts

Charts, like Grids, play a central role in a lot of financial & insurance software. You see the data in lists and you visualize it in charts. Ext JS 3 used Flash, but now they use SVG (and I think VML in some cases) and no canvas. This ensures it’ll work without a plugin. The downside is you can’t print yet. They’re working on it, and there are a lot of server-side plugins where you can easily wire up the chart data to a button to have the server assemble a PNG or PDF for you to print so solutions do exist.

You’ll notice, too, that they are interactive. You can customize a lot of their display + tooltips I believe to degree using templates (I’ll go over that later). They also have the standard animation built in as well. From a sales perspective, these charts  + the grids demo very well.

While we at WASI have our own Dashboard, it’s nice to know you can accomplish the same style of application and product using Ext’s components to create Dashboards.

Everything Else

Everything else is what you’d expect your team would need from Tabs to layouts to forms. The layouts are a little tricky if you’re not a CSS/DOM savant, but again, they abstract a lot of that from you (including that weird IE vs. the rest of the world padding measurement insanity).

There are a lot of little innuendo features too that make a big overall difference like all windows having the capability of being resized with a resizer, from all sides.

The components all have the standard DOM interaction events usually abstracted away so you just add an event handler for it, and you can create your own as well.

Yes, they have drag and drop.

Yes, they have a variety of ways to handle state, both in components and within your application itself.

Their drawing package uses SVG I think… not really sure why they aren’t using Canvas, but I’m sure they have their reasons.

Localization & Accessibility

They have localization and accessibility built in. I had some concerns during the fast track course because it seems the localization requires the application to be totally reloaded just to have text changes take effect. Meaning, there aren’t default framework bindings to a resource manager like you do in Java or Flex.

Either this was a bug or we couldn’t figure it out, but bottom line the components built into Ext JS do work with it, and you can see their text match the language; we just couldn’t get this to work at runtime, only compile time which is fine.

Event Model

Just a quick note on the event model. In Flash Player, we have the standard ECMA event model in terms of capture, target, and bubble phase. Since IE is backwards from the rest of the world and is in the minority camp, Ext JS flips it around and gives you their own Event class as well.

They key difference, however, is that interaction burdens are put on developer, much like Corona SDK does in Lua. If you want to handle an event, you need to return true to ensure it doesn’t bubble up and get run. I’ve seen other frameworks which just code gen the event.stopImmediatePropagation() and event.preventDefault(), but not here… and sometimes you need to return true from event handlers vs. calling the event methods. I haven’t figured out yet where the consistency is.

Anyway, a small thing, but good that they abstract the browser differences for you and provide a common event class + mechanism for you. The addEventListener and/or callback way of doing things will be extremely familiar. I believe they have an event bus, but it’s not like Robotleg’s Actor or PureMVC’s Notification system.

Styling & Theming

If you know CSS, you know how to style and theme a Ext JS application. Yes, that’s it. I unfortunately didn’t get to see, nor play with, a lot of the skinning in the Fast Track, just styling. I’m not sure how much of the image issues in various browser they abstract away since you’re using your own CSS styles.

Ext provides some template theme CSS files for you that you can modify, both built in classes as well as Ext JS ones. Combined with SASS/Compass, this really makes things a lot easier to handle. If you’re not familiar with SASS, there are just 4 things you need to know:

1. You write 1 CSS style, and it’ll generate the browser/vendor specific ones if any (1 line of SASS could be 3 lines of CSS)
2. You can utilize variables in your SASS code which helps dynamically create the appropriate CSS (like determining width/height on the fly, etc).
3. They actually treat it like css CLASSES… so all the inheritance feels like inheritance vs. some “I’m not really code, but I’ll pretend I know what OOP is”
4. SASS files get generated through Ruby’s Compass into CSS files for your project. You don’t write CSS; you write SASS and Ruby’ll regen the CSS for you.

Models, Proxies, and Stores 

Most Flash and Flex projects I’ve worked on have some form of Value Objects (classes that hold data that represent a server-side entity with additional data the client needs), Models or Proxy classes that allow the client retain data and state, and service classes that form the basis of the CRUD operations with the server.

In branching out into other languages/frameworks/platforms, it’s clear most have a large sub-system and abstraction layer upon this setup. Django has Model, Rails has ActiveRecord (or did… been awhile), Coccoa has CoreData (amongst other things), and Java has all sorts of things. Even with the large influx of Java, Flex never really got that kind of abstraction unless you went with the full Blaze DS / LiveCycle stack. It was abstracted away enough that you could build both non-Blaze/non-LC apps and not know the difference, but the point is that it was clear Adobe didn’t think that was a core part of most apps, or rather that particular part could be monetized in Enterprise only offerings.

Whatever the reasons, it’s a core need in a lot of applications nowadays. I need to persist data in RAM, on the client cache (whether SQL or cookies), and with the server. Using a Facade or Proxy to make this part not only easier, but allow client side validation, as well as being built into all components, is a huge win.

All you need to know is it’s like an ArrayCollection that can save both locally to disk as well as sync with the back end as well as work as a dataProvider to all your components. And yes, when the dataProvider updates, Ext JS will update your GUI controls. Yup.

Sencha has specific meanings in the Ext world between Model, Proxy, and Store so don’t necessarely think design pattern as each is completely different than the other.

JavaScript and Core Classes

While Ext JS primary value to Flex Developers is their component framework and browser insanity abstraction, they also have a plethora of helper/utility classes for JavaScript, CSS, and the DOM as well. You can actually just use Ext just with the DOM using their DomHelper, DomQuery, and your own JS stack if you wish as their helper classes alone are nice to have. Some of it is open source, not sure.

You’ll notice they actually have a layer over top of low-level functions of JavaScript like apply, they have their own version of Delegate (yeah, remember that bullshit? LIVE IT LOVE IT SUCKAZ AND THANK STEVE JOBS FOR IT), as well as a rudimentary binding syntax.

They support inheritance and package names for your components. Although I’ve never done Ext JS 3, it’s clear they are constantly improving because the syntax in 3 was nasty compared to 4.

Templates

I’m still learning about the world that is HTML Templates but suffice to say there are 2 main uses in Ext JS.

The first is custom components. Creating custom components in Ext JS is pretty straightforward; you just extend Component or Container, similar to Flex. However, they’re creating the DOM for you. What if you want to create your own HTML elements based on some strange PSD/AI design comp the company Crayon Pusher sent you? You can create your own HTML and have the data actually bound into it through a similar binding syntax we’re used to.

The second is item renderers. Any type of list, whether a small Container, or an actual component within a DataGrid itemRenderer can use a template. This allows quick creation of an itemRenderer without having to write a lot of code if you want to. Pretty sick.

MVC, MVP, MVVM, Your Mom

I’m not going to go into their MVC too much. Suffice to say, they like the Passive View and have no qualms with a Controller managing multiple Views (at least, that’s what the Fast Track implied, but they don’t have time in those types of courses to show you all avenues). Worse, their naming scheme assumes you have a flat file structure, so you don’t name something “MyModel”, just “My” because they’ll append a name to it anyway causing it to be MyModelModel. That, and I fail to see how a flat directory structure of model, view, controller will scale to an application of any reasonable size, even with Ext ability to use string namespace aliases to refer to your classes.

Keep in mind the instructor was only given 1 day to show 1 way to do MVC, and we didn’t get too much time to dive hard core into the internals. It was clear, though, none of us liked it, and already Flex 1 era framework ideas were churning to bring our ideas (and biases) to Ext JS.

What Sucks

1. Built on JavaScript, not CoffeeScript or Dart
2. They have horrible naming conventions for optimization purposes that no longer matter in todays world.
3. Tooling

The top of this post covers my major concerns about JavaScript for applications of this size. I’m not worried about performance, just team velocity compared to ActionScript. In my experiences with Lua and ActionScript 1, your velocity in the beginning tends to be faster, but you hit a plateau and the lack of strong-typing annihilates your teams velocity and ability to project accurately. Unit tests help, but are more of triage vs. a tool.

The naming conventions are bourne of an era no one should care about but they still do. We’re in a gray transition as browser makers bring the HTML Stack up to speed to build applications, so you can’t just full stop ignore all the lessons learned.

However, I’d argue, as would countless others, that code readability trumps performance concerns. This goes way beyond “premature optimization is the root of all evil”. On extremely large code bases, verbosity is a virtue. You want readable code that it is easy to understand, both for your team and for yourself.

Because of String lookup costs and the ability for JavaScript minifiers to only compress none-reserved words like “this”, countless techniques are used to obfuscate the code to compensate. That may have mattered back then, but it doesn’t matter now. Worse, though, is that the counter justification is that for larger code bases, you do in fact need to make every optimization you can which in turn I counter double fold:

1. Readable code trumps performant code.
2. If this is such a concern, why are we using this over Flex again?

More importantly, however, is for API development. API’s are given to people who you sometimes have no contact with, nor have any clue about their adeptness. Thus, you need to make it as easy and obvious as possible. For an Enterprise framework such as Ext JS, you’d think this would be top priority. It’s clear Ext JS has some highly experience web developers involved in developing the API. I respect their knowledge, but it’s time to change.

Examples include Ext.getCmp vs. Ext.getComponent, Ext.getEl vs. Ext.getElement, and baseCls vs. baseClass. It’s prevalent throughout the framework, and mixed in with good verbosity… which makes it worse.

Ext JS 4.5 clearly needs a name overhaul if they expect senior programmers to take it seriously. Considering it’s JavaScript with Ext JS’s alias support, you’d think this would be easier to do.

While Ext Designer completely blows away Adobe Catalyst, it still has some issues with regards to the code generated and code round trip/merging. While looking nice, the results are not always being correct. We had easily duplicatable issues where the code generated did not match what was in the IDE GUI. Still, an extremely useful tool for generating a lot of the needed visual component code.

Finally, the lack of out the box code IDE is a near deal killer, at least for strict TA’s we deal with at large organizations. I know Ted said they’re working on it. Without things like intellisense and code hinting, you’ll make it hard for people to take things seriously, as well as making it harder to learn for more traditional programmers who suddenly have none of their mature toolset available… like… Java Developers… who came to Flash… and left… and… then they created Flex Builder…

Critical Mass

It seems Sencha has all the hallmarks for success and reaching critical mass with Ext JS. This is the ONLY reason I and others have given them any attention. As previously mentioned, you need a mid-size company, with money, and a good product to have it adopted as a targetable platform for large organizations. Sencha seems to have all 3. That, and again their components are pretty cool.

As a consultant, their components, charts, Ext Designer, and website all demo nice as well making it easier to sell as an alternative to Flex.

Mobile Story

I won’t get into much here, but suffice it to say I’ve yet to see the mobile revolution impact our clients as much as it has impacted the Design Agency sphere. While both focused on consumers, for a lot of brands, smart phones and tablets are seen as a growth market, or the ONLY market, to reach certain demographics. All the statistics point to both the behavioral change of society using their devices to interact with the digital world vs. computers, yet those same statistics also who the desktop still being the leader in PRODUCING that content that the world consumes. Yet given that, the money doesn’t seem to be flowing in lock step.

As such, most initiatives have been “it’d be cool to blow some money on”, “we have to provide something for iOS, even if it’s crap”, and the most common “wait… this won’t make any money”. While I know there are a lot of device work going on out there, it’s just not in my consulting sphere, or if it is, it’s at a extremely low profitability area and thus not worth investing in.

For now, the same large interfaces that we make on desktops do not transfer at all to smart phones nor tablets. Just like how people started to taking their iPhone’s to work, and companies stopped installing company phones, and later with Macs so companies started offering Mac’s as well as PC’s… so too will this have to happen with tablets before large companies even have it on their radar as a focused set of initiatives.

Obviously this is dependent on industry type, but we’re just not seeing it in the financial/insurance area, only slightly in the media company area (which I might add are the only ones who seem to care about Android).

…but let’s pretend it DOES matter. Let’s say a Director wants to know your mobile portability story? What do you tell them?

With Flex it was pretty straightforward: 60% to 80% of your Service and Application Layer code could be ported, with no change, to have the same Flex desktop application work on iPhone, iPad, Playbook, and Android devices. The missing piece here, is the device’s web browser.

While I’m a huge/firm believe in a long term native future for devices, there are still many use cases when a consumer is in a hurry and hits a company’s website expecting it to work. Considering some Operators only allow some applications to be downloaded over Wifi vs. 3G, this makes the website more important since even 3G can act as Edge. If you’re a business traveller, regardless if you’re on AT&T or Verizon, you know exactly what I’m talking about.

2 key examples include Marriott.com and PapaJohns.com. I encourage you to play with both on your iPhone/Android’s web browser. You’ll notice they try to duplicate the native  look & feel paradigm. “It works like an app!”. Sometimes it works, sometimes it doesn’t. The responsiveness is the first thing you’ll notice that falls on it’s face. Apps just perform better, and consumers can tell. At least for now.

That’s not the point, though; notice how the functionality they need is forced to be simple because of the screen real estate being so small. There are a lot of use cases where even just a company mobile app is needed for its Desktop equivalent. Other times, you really do want to get as much information as possible from the user (a la Papa Johns). To me, Papa Johns is the perfect example of an application that an Enterprise would want on a phone, then they realize at the design/analyze phase/outset that most funtionality couldn’t be done, but shove in as much as they could anyway.

If you’re forced to comply, what do you do?

You use Sencha Touch.

Now the same story applies like it did in Flex with 1 key differnence: Using PhoneGap, you can now target everything. All devices including their web browser, all while sharing 60% to 80% of your Service and Application Layer code. In theory… I’ve heard a lot of negativity towards Sencha Touch v1, but apparently v2 is a lot better, haven’t investigated the code yet, just testing the components on devices.

Pretty cool, huh?

Conclusions

If you haven’t read João Saleiro’s article, and you’re a Flex Developer, I encourage you to do so. I’ve reached many of the same conclusions as he did. The comments also have a lot of good info as well. The most important being team velocity. Time is money. This isn’t important to just large companies, either.

There are a lot of things we can do better and faster in Flex for large scale applications. It doesn’t matter, however, for 2 reasons. First, while from a technical perspective Flex’ future is bright, from a marketing and business one it’s over. The next 5 years are the sunset. Like FORTRAN and Cobol, companies invested billions, and those applications that run parts of their business don’t magically disappear overnight. Second, even if we know it may be a better solution for some use cases (many objects like Visio, highly dynamic interactions w/many redraws, compositing images & video, etc), we’re ultimately beholden to what our clients want, even if we know what they need is different. Thus we have to do both: Flex and Ext JS.

Based on all of read and seen, next to Dojo, Ext JS is the best solution to utilize for HTML5 web application development if you are a Flex Developer dealing with large applications and teams. Their components, documentation, mobile story, company size, and growth show that they are close to reaching critical mass where it can be adopted by large companies as a viable product to use for things traditionally done by Flex. Additionally, a lot of what you already know carries over to Ext JS quite well. If you’re like me, you just need to freshen up on your DOM and CSS knowledge, and re-examine JavaScript with fresh eyes compared to the AJAX revolution bs of 7 years ago.

Furthermore, unlike other server-side solutions, Ext JS lends itself quite well to hybrid solutions with Flex being used for certain use cases and the HTML stack for others, all on the same project. You and/or your firm having the capability to provide both is immensely valuable. Utilizing Sencha Touch and PhoneGap, you now have multiple mobile options in your tool belt as well.

For the record, I still believe using Ext JS will take longer and have a harder to track velocity compared to using Flex. That said, I’m still diving into Ext JS with a positive attitude.

Good luck!