Neoclassical Narratives

Principles and Practices of Cloud Applications

2017-08-26T15:17:00.001-05:00

This article summarizes the essential principles, patterns, and practices of cloud application architecture and design that can increase organizational innovation and decrease long-term system maintenance costs. My primary sources are the architecture teams at Amazon Web Services and Microsoft Azure as well as my professional experience daily applying these ideas to production systems. Subsequent articles will focus on processes and technologies specific to AWS and Azure.

Principles

Patterns

Practices

Scalability

Decomposition

Partitioning

Stateless

Elasticity

Caching

CDN

Queue Worker

Pipes and Filters

Encapsulation

Materialized View

Eventual Consistency

Understand customer SLA for performance.

Measure and profile performance with load benchmarks.

Partition and decompose workloads into discrete units.

Partition around data, network, and compute limits.

Design for horizontal scalability (scaling out/in vs. up/down).

Ensure applications and services are stateless.

Avoid client affinity and server-side session state.

Minimize coordination and shared state.

Queue I/O and CPU intensive requests as background tasks.

Distribute background tasks across multiple workers.

Cache items that don’t change much.

Use CDN for caching static data.

Reduce chatty interactions between components.

Build golden component images using Docker.

Leverage PAAS auto-scaling features with golden images.

Consider compression and binary format for DTO transfer.

Optimize SQL indexes and queries.

Consider document DB or de-normalizing data model.

Avoid locking database resources.

Prefer optimistic concurrency and eventual consistency.

Minimize time that connections and resources are in use.

Minimize number of connections required.

Resiliency

Redundancy

Load Balancing

Retry

Circuit Breaker

Replication

Healthchecks

Telemetry

Understand customer SLA for availability.

Analyze system to identify failures, impact, and recovery.

Use redundant components to minimize single point of failure.

Use load balancing to distribute requests.

Handle transient failures with limited retries and backoff.

Handle persistent failures with circuit breaker that falls back to reasonable action while dependency is unavailable.

Use multiple availability zones.

Monitor health of dependencies and endpoints.

Checkpoint long-running transactions.

Design for failure and self-healing.

Understand replication methods for data sources.

Automate persistent data backup.

Document failover/failback processes and test them.

Throttle excessively active clients. Block bad actors (DDoS).

Perform fault injection testing to verify system resiliency.

Security

Defense in Depth

Least Privilege

Traceability

Federated Identity

Gatekeeper

Compartmentalize

Apply defense in depth; secure all resources - not just edges.

Secure weakest link. Trust reluctantly and verify. Fail securely.

Pay attention to data privacy and residency requirements.

Protect data at rest (storage encryption) and in transit (SSL).

Mitigate DDoS using cloud platform’s network layer.

Enforce ACL’s at network, application, and data layers.

Conduct vulnerability analysis and penetration tests.

Manage keys carefully and secure with hardware tokens.

Use SSO, multi-factor authentication, and federated identity.

Use anti-virus and anti-malware for network and host nodes.

Simplify BCDR through PaaS centric, automated backup and recovery.

Integrate diagnostics of network, application, and data layers to have monitor system and correlate enterprise intrusions.

Prefer connectivity from cloud to on-prem resources using dedicated, private WAN links vs. VPN tunnels over public links.

Application Design

High Cohesion

Loose Coupling

Single Responsibility

Open/Closed

Interface Segregation

Dependency Inversion

DDD

CQRS

RESTful Web API

Messaging

Design with the organization goals and end-user in mind.

Design for evolution and change.

Prefer loosely-coupled components whose communication is asynchronous that can evolve, heal, and scale smarter.

Separate infrastructure logic from domain logic.

Prefer RESTful Web API’s for external communication.

Prefer asynchronous messaging for internal communication.

Management

Telemetry

Automation

Source Control

Agile

Design for IT Ops (Deploy, Monitor, Investigate, Secure)

Document system release process and use change control.

Automate system build and deployment processes.

Implement logging and alerting into systems.

Instrument to analyze root cause of errors.

Instrument to monitor availability, performance, and health.

Standardize log formats and metrics.

Inventory, inspect, and audit cloud assets.

Use distributed tracing (asynchronous, Correlation ID).

Version and control configuration like other system artifacts.

Use Agile project methodology for iterative development.

References

● https://aws.amazon.com/architecture/

● https://docs.microsoft.com/en-us/azure/architecture/

7 Habits of Highly Effective Architects and Developers

2017-07-04T21:50:00.000-05:00

Independence (Self)

1. Understand Technology Holistically. Delivering technology solutions requires more than choices about just a programming language, system library, or infrastructure provider. It means having a comprehensive vision aligned with organizational capability and thinking about security, performance, databases, operating systems, infrastructure (cloud vs. on-premise), availability, resiliency, networking, user interfaces, device access, monitoring, reporting, testing, product/project management, testing, support, customer requirements, change management, and much more. The whole matters beyond the sum of the parts.

2. Get Things Done. Balancing principle and practice is an art, not a science. This balance is situation dependent; focusing on delivering value to customers (doing the right thing at the right time) whilst giving attention to the system’s evolving architectural quality (doing things the right way). Where there is technical debt, be mindful and make plans to pay it off per some reasonable schedule. Prioritize what is important and weight tasks by “bang for the buck”; some call this being Agile or common sense. For frequently urgent or distracting matters, resolve the root cause so primary focus can return to what is truly important.

3. Work Efficiently. Use advanced CASE and IDE tools. Automate repetitive tasks such as regression tests, builds, and deployments. Communicate effectively by email, telephone, IM, and also in person by facilitating and presenting information in groups. Use wikis to share information; use online backlogs to manage projects and tasks across the team.

Interdependence (Others)

4. Care about Customers. Customers and the organizations we serve give us a purpose and are the reason we are compensated for what we do. We must listen to users, show empathy, understand their requirements, especially those expressed and obvious, but also surface those in which intent is uncertain and less well known. Once we understand, then we also need to be understood through careful communication about our designs and projects in language and symbols that are precise, mostly visual, and jargon free.

5. Collaborate. The concept of the lone coder as a hero is mostly a myth. Big, important problems in most organizations require collaboration amongst different stakeholders and a win-win mentality rooted in the common values and practices of respect, humility, integrity, change, articulated decision rights and incentives.

6. Keep Things as Simple as Possible. Systems and People (and Life) can be complicated, but it does not have to be. By emphasizing core values over rigid rules, we can allow important systems and relationships to remain strong, flexible, and still aligned around what is important.

Continuous Improvement

7. Passion. Great developers and architects love what they do. You cannot teach it, fake it, or hide it. They enjoy sharpening their saw, improving themselves and others around them.

References

https://blogs.msdn.microsoft.com/johnwpowell/2008/05/22/the-7-habits-of-highly-effective-developers/

https://www.stephencovey.com/7habits/7habits.php

http://www.goodprofitbook.com

http://tyrisias.blogspot.com/2016/12/principles-and-practices-sdlc.html

7 Habits of Highly Effective Architects and Developers

Book Summary | The Coach

2017-02-13T22:24:00.001-06:00

This article summarizes the key takeaways of the The Coach, Creating Partnerships for Competitive Edge by Steven Sowell and Matt Starcevich. Rich with useful, psychological insight into how to genuinely improve relationships between leaders and employees in service of organizational advantage and excellence, I highly recommend reading this evidence-based book and putting it into daily practice.

8 Step Model

Step 1: Be Supportive … assist, empathize, understand, listen, encourage

Step 2: Define the Topic and Needs … focus on present, 1-2 items/session

Step 3: Establish Impact … talk costs and benefits, mirror as role model

Step 4: Initiate a Plan … who, what, when, where; set SMART goals

Step 5: Get a Commitment … get clear answer and verbal/written agreement

Step 6: Confront Excuses and Resistance … focus on what can be done

Step 7: Clarify Consequences, do not Punish … focus on +, explain -

Step 8: Do not Give Up … persist, establish follow up time/place

5 Levels of Coaching

Linking and Bonding - spend most of your time here, daily

Reviewing - monthly 1-on-1

Mentoring - continuous

Confronting - if and only if needed

Choosing and Developing - quarterly, annually

Management Models

Adversarial / Hierarchy - traditional, Tayloristic view, control

Patriarchy / Matriarchy - ineffective

Silence - ineffective

Entrepreneurial - building effective partnerships with employees (Koch MBM)

Definition of Partnership

Teamwork

Common interests and goals

Equal status

Mutual consent

Respect

Honesty

Process of Guided Change

Stage 1: Preliminary Events … detect, pinch + pressure, decide

Stage 2: Support-Initiate using 8 Step Model

Stage 3: Follow Up Coaching … refine + reinforce, feedback + support

Supporting Skills

Be forward looking … focus on improving future, not pathology about past

Give employees credit for recognizing the problem and designing soln.

Start light, indirect in communication style (ask questions, listen, probe, restate, encourage); then and only then get direct if needed.

Focus on the concern or problem … not the person.

Set clear expectations (Koch RRE).

Shift to silence when done speaking to put ball in employee court.

How to Better Coach

Be assertive.

Be realistic.

Trust the process.

Coaching

Is a process… not an event.

Coaches are catalysts and conductors … not commanders.

Sports

John Wooden .. UCLA College Basketball Coach

Barbara Bramblett - my tennis coach

Positive Development

Process

Trust employees more.

Give them more responsibility and variety.

Defer to them more.

Surrender more control to them as mutual business partners.

Allow them space and creativity.

Principles and Practices | Web API

2017-01-02T13:34:00.000-06:00

This article enumerates the best principles and practices for designing, deploying, and maintaining successful Web API’s that can drive organizational innovation forward. This document has borrowed and consolidated ideas from Apigee (Bryan Mulloy), Microsoft, and Mulesoft (Mike Stowe); I highly encourage the reader to also review these primary sources. I will use the fictional Acme company as an example throughout this article.

Category	Principles and Practices
Requirements	Understand your users. What are the use cases? What technologies/services will be used? What are the business resources that need to be managed? What are their entity relationships? Acme Answer: Users are digital Builders and the key resources are Bricks
Design Strategy	Think long-term. Not just release 1. But think 2-3 years ahead. Ensure the API is platform independent to maximize API portability and usage. Be stateless to maximize scalability, increase cacheability, and reduce client/server dependencies. Be consistent. Be flexible. Keep It Simple and minimize complexity throughout.
Prefer Nouns, not Verbs	Prefer plural nouns, instead of verbs. http://acme.com/api/bricks [preferred] vs. http://acme.com/api/getbricks [avoid, since this is more verbose and leads to complexity] Use verbs if you are implementing calculations or transformations that are unrelated to resources.
Map CRUD to HTTP verbs	DELETE, GET, PUT, and POST requests should be idempotent. http://acme.com/api/bricks Create => POST Read => GET Update => PUT Delete => DELETE
Map Errors to HTTP Status	Handle errors consistently, at least 5 of the 70-ish HTTP response status codes. 200 - OK 400 - Bad Client Request Format 401 - Unauthorized Client Request 404 - No Resource 500 - Server Error 501 - Not Implemented 503 - Service Unavailable Map exception and detailed error messages into the response payload.
Version API	Never release the API without a version. This is part of managing change and being flexible. Prefer versioning in the base URI to be explicit instead of a request query or header parameter. Prefer simple ordinal numbers over dot notation and timestamping. Prefer v1 instead of v1.0 or v2016-01-01. http://acme.com/api/v1/bricks
IO Formats	Default format for input and output should be JSON. Support multiple input and output formats. Optionally support XML for input and output as well CSV for output. Support content types specified in the URI: GET /api/v1/bricks.json GET /api/v1/bricks.xml Support content types specified in request header: Content-Type: application/json
Collection Navigation	Provider filtering, sorting, field selection, and pagination of collections. GET /api/v1/bricks/?color=red GET /api/v1/bricks/?sort=model GET /api/v1/bricks/?limit=50&offset=10 GET /api/v1/bricks/?fields=id,model,color
Consolidate	Consolidate all API services and requests into one domain. Example: http://api.acme.com/v1/
Document	Document your API and publish the specification online. Use Swagger or RAML.
Security	Authenticate and authorize all users. Validate input requests. Prefer declarative security on individual methods of service endpoints. For private intranets using Active Directory, assign permissions to roles consisting of AD groups. For public extranets, consider using OAuth 2 and SSL. Throttle users that exceed API limits and generate 503 errors with Retry-After headers.
Deploy	Log requests for troubleshooting and customer metering. Support multiple channels for customer feedback: email list, online forum, Slack channel, etc. Monitor API health, activity, and usage in Production. Consider using 3rd party products for enterprise API management and standardization.

References

● Apigee

● Microsoft

● Mulesoft

● Rest Cookbook

● Mathieu Fenniak Checklist

Excel Add-in Development | Portability

2016-12-28T07:20:00.001-06:00

Designing and developing portable Excel add-ins across different versions of Microsoft Office (2000-2016, x86-x64) is an important goal for keeping maintenance costs low whilst enabling you to grow and support as broad a Office customer base as possible. This article describes how to do just that by combining two open source .NET libraries: Excel DNA and Net Office.

Excel DNA is a .NET library whose API’s simplify writing COM-free Add-ins with User Defined Functions, custom Ribbons, RTD components, and VBA script friendly API’s for Office task automation. The project started by Govert van Drimmelen in 2005 has a broad user community across the financial and scientific domains.

Net Office is a .NET library whose API’s allow one to target multiple versions of Office 2000-2016 without referencing the platform specific PIA and VSTO assemblies as well as managing the COM proxy objects. The project started by Sebastian Lange in 2011 also has a large user community across the Office platform. The Net Office API allows one to interact with the Excel object model (e.g. workbooks, worksheets, ranges) in a Office platform independent manner.

Architecture

Implementation

1. Create a .NET class library in your preferred language (e.g. C#, F#, VB.NET).

2. Add references to NuGet packages of Excel DNA and Net Office.

a. ExcelDna.Integration

b. ExcelDna.Addin

c. NetOffice.Excel

3. Define a class that implements ExcelDna.Integration.IExcelAddIn interface as well as the System.IDisposable interface.

a. Implement the Singleton pattern.

public class MyAddIn : IExcelAddIn, IDisposable

{

// singleton variable

private static MyAddIn m_Instance = null;

// instance variables

private object m_ComApplication;

private Application m_ExcelApplication;

private bool m_Disposed = false;

static MyAddIn()

{

m_Instance = new MyAddIn();

}

public MyAddIn() {}

public static MyAddIn Instance { get { return m_Instance; } }

}

b. Implement the AutoOpen and AutoClose methods. These methods are invoked when your Addin is first installed and then un-installed.

c. Implement new OnActivate and OnDeactivate methods. In the former, get a reference to the root Application object from Excel DNA and connect to a Net Office Application wrapper, and voila, you now have programmatic access to the Excel object and event model through a platform independent API.

internal void OnActivate()

{

if (m_ComApplication == null)

{

m_ComApplication = ExcelDnaUtil.Application;

}

if (m_ExcelApplication == null)

{

// glue NetOffice + Excel DNA!

m_ExcelApplication = new Application(null, m_ComApplication);

}

internal void OnDeactivate()

{

Dispose();

GC.Collect();

GC.WaitForPendingFinalizers();

GC.Collect();

GC.WaitForPendingFinalizers();

}

d. Implement the Dispose method and release references to the root Net Office and Excel DNA application objects. This will prevent memory leaks and eliminate the “phantom” Excel process after your program exits.

public void Dispose()

{

Dispose(true);

}

public void Dispose(bool disposing)

{

try

{

if (m_Disposed)

{

if (disposing)

{

if (m_ExcelApplication != null)

{

m_ExcelApplication.Quit();

m_ExcelApplication.Dispose();

}

if (m_ComApplication != null)

{

Marshal.ReleaseComObject(m_ComApplication);

}

catch { }

finally

{

m_Disposed = true;

m_ExcelApplication = null;

m_ComApplication = null;

}

4. Define a class that either extends ExcelDna.Integration.CustomUI.ExcelRibbon or ExcelDna.Integration.ComAddin.

a. Mark the class with several System.Runtime.InteroperServices attributes.

i. Mark ComVisible as true.

ii. Set the ProgId as some fixed string value.

b. Override the OnConnection and OnDisconnection methods and invoke the Addin’s OnActivate and OnDeactivate methods through the singleton.

[ComVisible(true)]

[ClassInterface(ClassInterfaceType.AutoDispatch)]

[ProgId("Athena.Excel.Sample.MyAddIn")]

public class MyRibbon : ExcelRibbon

{

private MyVBAFacade vbaProxy = new MyVBAFacade();

public override void OnConnection(object comApp, ext_ConnectMode ConnectMode, object objAddin, ref Array custom)

{

base.OnConnection(comApp, ConnectMode, objAddin, ref custom);

if (objAddin != null)

{

dynamic excelAddin = objAddin;

excelAddin.Object = vbaProxy;

}

MyAddin.Instance.OnActivate();

}

public override void OnDisconnection(ext_DisconnectMode RemoveMode, ref Array custom)

{

base.OnDisconnection(RemoveMode, ref custom);

MyAddin.Instance.OnDeactivate();

}

public void OnHello(IRibbonControl control)

{

MessageBox.Show("Hello, world!");

}

5. Use the Addin.Instance.Application reference to interact with Excel API through Net Office in a platform independent manner that works across Microsoft Office 2000-2016. Example scenarios include the following:

a. Register for Excel events such as file open/close and macro re-calculations.

b. Manipulate Excel cell ranges in response to user events and custom functions.

c. Perform context specific activity based on the active Excel workbook or cell.

d. Prompt the user for input from the Excel range reference dialog box.

e. Set the Excel status bar.

Principles and Practices | SDLC

2016-12-15T06:26:00.001-06:00

This article summarizes some of the core principles, best practices, and my common sense for governing the Software Development Life Cycle. These SDLC ideas are applicable for both products and projects. I look forward to discussing these ideas with the wider community.

Category	Principles and Practices
Feasibility	Identify the product/project investors, business sponsor, and key stakeholders. Establish economic model with costs and benefits spanning average and best/worst cases. Write a clear mission and vision statement.
Requirements	Document requirements for Minimum Viable Product (MVP). Document user stories, data model, 3rd party constraints, as well as the why, not just the what. Prototype UI and MVP concepts; get feedback from key users and domain experts.
Project Management	Define delivery of the project or product into large milestones and smaller iterations. Use 3-4 weeks per iteration and sprint. Plan sprints as a team. Schedule daily scrums and weekly meeting to review progress, obstacles, and scope. Define change control process that connects stakeholders with cost/benefit tradeoffs of change. Create work breakdown structure; define and sequence activities. Get labor and material cost estimates from engineering - not management or marketing. Establish a single point of accountability for technology, process, and business matters. Plan and distribute communications. Manage stakeholder expectations. Report performance. Identify top 10 project/product risks. Monitor and communicate risk assessment throughout. Identify core team of 3-5 people (e.g. size of large pizza) including business, engineering, and QA. Publish planning documents for managing costs, risks, QA, team, procurement, and requirements.
Architecture	Define major building blocks including their responsibilities and interfaces (Miller’s Law: < 9). Visualize the solution architecture for shared understanding using UML diagrams.
Design	Single Responsibility - components should have only 1 reason to change and exist. Encapsulation - share only what is needed; hide the details. Cohesion - gather and release related components, functions, and data together. Dependency Inversion - depend on abstractions, not concretions to manage change. Simplicity - make components/systems as simple as possible; minimize complexity. Stability - use stable abstractions, dependencies, and interfaces. Consistency - use consistent naming conventions, code styles, etc. DRY - Do not Repeat Yourself. YAGNI - You Ain’t Gonna Need It. Things Change. Plans change. Avoid gold plating. KISS - Keep It as Simple as poSsible. But no simpler. Refactor during maintenance iterations. Design does not end in one phase. Consider domain specific languages to represent flexible logic and rules. Consider reflection, attribute-based programming for dynamic feature surfaces. Consider microservices to reduce monolithic system into modular components. Consider concurrency and scaling to multiple processors and machines. Consider portability and migration to other platform/infrastructure environments. Consider localization and internationalization.
Security	How is the solution secured? Consider users, data, and system itself. How are users authenticated and authorized? Does data need to be encrypted in storage and transmission? Grant minimum privileges to users and service accounts. Use enterprise identity management systems (e.g. Active Directory, ) that federate. Avoid inventing your own user/group/password management and encryption systems.
Database	Is ACID necessary? Then perhaps SQL/RDBMS. Or does scalability suggest NoSQL? Visualize logical entities and their relationships using diagrams. Use primary keys and foreign keys for referential integrity. Use default and check constraints for column data integrity. Use indexes for large tables (1M+) and partitions for larger data sets (1B+). Use NoSQL, document oriented databases for very large data sets (1T+). Avoid cursors, triggers, and dynamic SQL. Use them sparingly and consciously. Secure access through groups and roles. Plan for growth and disaster recovery. Stress test for performance and reliability.
Artifact Control	Manage source code, binary references, and documentation using Version Control System. Prefer GIT; TFS and Subversion are OK though. Secure access to artifacts through groups and roles. Plan for several people working on system over time, perhaps at same time. Source code is not owned by one person; it is collectively owned by the team.
Quality Control	Code defensively. Check parameters. Use assertions. No broken windows. Are there tests for Correctness? Integration? Performance? Security? UX? Define automated unit tests such as MSTest, NUnit, jUnit, or RUnit. Assign manual beta testing driven by checklists to independent group of power users. Review designs before coding. Review code before production deployment.
Devops	Where will the solution run? desktops? Mobile? On Prem? Cloud? How will you deliver system updates and changes over time? Maintain separate environments for Development, Test, and Production. How do you log system activity especially errors and end-user usage? Automate builds and deployments using VSTS, Chef, Octopus, PowerShell, bash, etc.
Support	How will users contact you regarding features and defects? Track past, present, and future work in an online backlog. Use JIRA, VSTS, or a similar system. Avoid spreadsheets and email.
Customers	Delight users. Under promise and over deliver. Prioritize system feedback from power users who influence others and are passionate. Key users should be part of project and product teams and available - onsite, daily.
Peopleware	Align team member interests with work assignments. Create workspaces that support thoughtful, focused, un-interrupted work. Avoid open offices. Establish accountability and responsibility for decisions by making people put skin in the game. Take time to celebrate achieving milestones and performing post-mortems.
Metaphors	Cone of uncertainty - explore risk areas and learn to reduce uncertainty over lifecycle. Triangle of constraints - cost, scope, and quality. Pick 2 out of 3; it’s tough to have it all. Tracer bullets - use prototypes and feature spikes to find the moving target in the dark. Microstones - track progress on smaller units work to keep team focused within milestones.
Proverbs	Work with users to think like a user. Don’t gather requirements like scattered seeds; dig for them like buried treasure. You will fail, if there is no plan, no one following plan, and no change to plans when things change. Abstractions live longer than details. Test early, often, and automatically. Provide options; don’t give lame excuses. The word of someone else’s mouth sells better than opening yours. Good architectures with conceptual integrity should manage complexity and change. Do not assume something to be true; try to prove it. Sign your work; take pride in quality and craftsmanship. If you found it painful more than twice, then automate it. The cost of “quick and dirty” often lasts longer and gets dirtier than the quick win’s benefit. The best and the perfect are the enemy of the good. You don’t make money by developing software, but by delivering and selling software. Leverage 80/20 rule - finish 80% of requirements before design and 80% of design before coding.

References

● Feasibility/Requirements

○ Lean Enterprise by Humble, Molesky, and O’Reilly

● Design/Construction

○ Extreme Programming by Kent Beck

○ Pragmatic Programmer by Andrew Hunt and Dave Thomas

○ Code Complete by Steve McConnell

○ Design Patterns by GoF

● DevOps

○ Release It by Michael Nygard

● Project Management

○ Essential Scrum by Ken Rubin

○ PMBOK by Project Management Institute

● SDLC

○ Software Project Survival Guide by Steve McConnell

○ Peopleware by Tom DeMarco and Tim Lister

○ 12 Steps to Great Code by Joel Spolsky

○ Principles and Practices of SDLC

Excel Add-in Development | Framework Criteria

2016-12-04T16:39:00.001-06:00

Although the desktop may be slowly dying as phones, tablets, and other devices encroach; many businesses and consumers remain loyal fans and active users of Microsoft Office and Excel. According to Microsoft, there were 1.2 Billion users of Office as of March 2016. Excel’s core capabilities include a flexible tabular data representation indexed by rows and columns, a charting module, a catalog of built-in functions, and a rich programming model using VBA. For heavy, data intensive commercial and academic users of Excel, there is a need to extend Excel’s core capabilities with custom calculations, new data sources, real-time refresh, and embedded GUI’s.

In this series, I will be covering several topics.

Excel Addin Framework Evaluation Criteria
UDF
Embedded GUI
RTD
Portability
Deployment
Troubleshooting

There are several criteria when evaluating which Excel addin framework to use.

Criteria	VSTO	Add-in Express	Excel DNA	PyXLL
Platform	Microsoft COM, VSTO, C/C++, .NET, VBA	.NET	.NET, VBA	Python v2.3-3.6
Price	MSDN	Commercial. $500-900/dev/yr	Free	Commercial. $100-200/dev/yr
Portability	No. PIA version dependency.	Has portability layer for O2000-2016.	Use with Net Office to support O2000-2016.	Supports O2000-2016.
Open Source	No	No	Yes	Yes
Support	Yes	Yes.	Yes	Yes
UDF	Yes. Native	Yes	Yes	Yes
Ribbon	Yes. Native	Yes	Yes	Yes
RTD	Yes. Native	Yes	Yes	Yes
GUI	Yes. Native	Yes	Yes	No
Office Breadth	Yes. Native	Supports Excel, Word, and Outlook	Only Excel	Only Excel
History	1997	1998	2005	2010

If you are looking to build an Excel product suite or family of add-ins, then I strongly encourage you to examine Add-in Express or Excel DNA. Both are fully featured and have broad language support (unlike PyXLL), and both are much easier to use than the native Microsoft VSTO libraries. Both have active user communities with 1000’s of developers each.

If you intend to support other Office products such as Word or Outlook and are also not comfortable with using open source software, then choosing Add-in Express makes wise sense from a long-term maintenance, TCO perspective.

If you are budget conscious and are developing a PoC or MVP for just Excel, then choosing Excel DNA and Net Office is a solid starting point that you can build upon.

If you intend to develop Excel add-ins which use native, platform features that are idiosyncratic to a specific version of Office (which I do NOT recommend unless it offers durable competitive advantage to your customers), then use VSTO.

Book Summary | Scalability Rules

2016-12-03T21:00:00.000-06:00

This article is a reboot of my blog. Posts will consist of book reviews, informal musings on economics, history, and software, as well as more formal, didactic articles on technology.

Here is a book summary and review of Scalability Rules by Abbott and Fisher. Practical. Concise. Packed with good advice on design, infrastructure, and organizational processes. The latest edition includes stories that give context to the rules. Highly recommended.

Category	Rule
Design	Do not over engineer the solution. Design scale into solutions: design @ 20X, implement @ 3X, deploy @ 2X Avoid single point of failure. Avoid sequencing components and systems in synchronous series. Strive for statelessness. Watch out for server affinity. Communicate asynchronously as much as possible (prefer pub-sub to RPC). Design your application to be monitored.
Reduce	Reduce object usage including DB, DNS, sockets, images, CPU, MEM, DISK, etc
Cache	Cache static, infrequently changing information and objects appropriately. Use web page cache, image cache, application reference data cache, etc.
Scale	Design to clone things on commodity hardware and network load balancer. Design to split different and similar things. Design your solution to scale out. Use AKF scale cube. Scale out your data centers. Design to leverage the cloud, but be wary of scaling through 3^rd party. Details matter.
Platform	Use database appropriately. Need ACID (Atomic, Consistent, Isolated, Durable)? Use firewalls appropriately. Use log files actively. Do not double check your work… do not re-read your data after write. Stop redirecting traffic. Leverage Content Distribution Networks (CDN). Use expire headers in HTTP requests to reduce duplicates for static data. Purge, archive, and cost justify storage.
Process	Test. Measure. Rinse. Repeat. Learn aggressively and especially from failure through team discussion. Do not rely on QA to find mistakes. Design for rollback of code. Be aware of costly relationships and dependencies (networks, databases, etc).
Database	Understand object relationships. Use the correct database lock. Avoid multi-phase commit. Avoid select for update. Avoid select *. Separate business intelligence from transaction processing.
General	Do not do anything more than once. Do not do anything that is unnecessary. Get as much as in one request as possible.

Cube Restore Chokes Without SQL Server 2005 SP2

2008-07-09T23:35:00.005-05:00

I just rebuilt my development machine's SQL Server 2005 instance and forgot to re-install SP2. One of the quirky reminders to "do the right thing" and update/patch the base installation is transferring hypercubes between instances of SQL Server 2005. See the screenshot on the right for an example of what the hand slap looks like.

Veterans have experienced this minor flesh wound before.

http://weblogs.asp.net/jgalloway/archive/2007/06/21/sql-server-analysis-services-quot-errors-in-the-metadata-manager-quot-when-restoring-a-backup.aspx

SEC Approves XBRL Rule

2008-06-29T19:32:00.004-05:00

If accounting is the language of business, then the SEC's recent vote requiring public companies to electronically file financial documents using XML within 3 years means that XBRL is a leading candidate as the language of e-business.

What does it mean for the average agent (i.e. investor, creditor, regulator, etc) in the financial ecosystem? XML tags data, and XBRL defines financial taxonomies. Combined with gradual convergence of GAAP and IFRS, this ruling should catalyze financial information system integration.

http://www.sec.gov/news/press/2008/2008-85.htm
http://www.xbrl.org/

Dunking Data

2008-06-19T20:25:00.005-05:00

Only a former highschool basketball statistician would get excited amidst the NBA Finals hype to report about the National Basketball Association's state-of-the-art information system that monitors and reports its games.

The scoreboard, shotclock, referee whistles, and floor lighting system all connect to a LAN of laptops operated by staff who tap dance with the data before it is sent to the arena's Large Monitors and media outlets. The data is sent onto NBA HQ in Secaucus, NJ where it is published to nba.com. Video feeds are also tagged and fed into a database that is used by coaches to plan games and the league to review calls. How cool is that?

http://www.eweek.com/c/a/Infrastructure/Data-Infrastructure-Drives-NBA-Finals/