BlackInkBottle's Ink

Microsoft's Search Strategy

blackinkbottle — Sat, 11 Apr 2009 05:00:00 GMT

This is the experiment. Randomly-selected participants are asked to score the quality of results from various search engines (the below details may contain some speculation).

One group (A) are presented with results from different search engines but not told which search engine has generated the respective results. Results from Microsoft and Google are presented.
Another group (B, or perhaps the same group at different time?) are told the results (which actually are generated from search engine X, or Microsoft) are by Google.

This is "blind" experiment. Group A is considered to the control group of Group B (I think so). The result is interesting: for Group A, there is no noticeable difference observed for perceived quality of search result between Microsoft's and Google's; for Group B, the perceived quality of search result becomes greater by simply knowing they are from Google. (Story and some commentary is here)

It is this result that leads Microsoft to believe the branding is the root of the problem (rather than the search quality). In coming June Microsoft is said to launch its rebranding campaign (for search) and the latest version of the search engine. The campaign is estimated to cost $100million.

Let us pause for a while.

No doubt, Microsoft perhaps has some misstep in branding its various recent products (eg Live Messenger, Live Search, www.live.com, Live Meeting, Live Search Maps, Windows Live ID, Office Live, Office Live Workspace, XBox Live and the list goes on), as written here in 2007 May, "maybe Microsoft should really think hard on its branding strategy, Live is a boring word, a confusing world".

So branding is indeed overdue. However some possible names rumored on the Internet (including those codenames disclosed) are not so engaging:

Kumo (codename)
Kiev (codename)
Bing
Hook

I will think of a lot of question marks (???) if any of them makes into the final champion. That might just create more trouble and confusion. Indeed it is difficult to incept a lasting brand. Many succeeded, Youtube, Facebook; but a lot more became derailed. However, Microsoft need make sure she can afford the failures one after another.

Second and more importantly, is the branding the root cause? If the Group A tells no difference, why would people make a willing switch (if not given any other external force?) It is difficult to break consumers' habit or preference of search engine (many tried and failed miserably), especially if the new comer does not produce any significant advantage and the old one does not make apparent fault.

In the end, or a "iPod" is created when the others lack the branding (unlikely for search), or the proflieration and lock-in of inferior products (Microsoft has already done a few times, but not again, unlikely for search), or reaching a higher level or creating a new battle space. So there is no option but the third.

Disclaimer: Btw, I am long on MSFT at the time of writing.

我喜欢这样的天气

blackinkbottle — Tue, 31 Mar 2009 16:05:00 GMT

"我喜欢这样的天气，不是烈日盛夏，就是严冬酷寒，不是倾盆大雨，就是风雪交加。"

激烈对峙,强烈对比是他电影的一大特色.镜头的色彩,光影,场景构图, 人物构图无不为电影的气氛服务。"我在拍摄中习惯由手绘分镜开始.这个使脑海中的影像具体起来，手绘分镜应不断地做出修改，因为电影是有生命的". 这个生命,我想,就是要把"人性的每一面都投射出来,最难以用逻辑解释的部分".

Once upon a time in America, once again

blackinkbottle — Sat, 28 Mar 2009 07:03:00 GMT

I guess a poem about passage of time is better appreciated with more passage of time (230 minutes runtime)... I watched it numerical times and I found it short each time I watched it again :) Friendship, love, gain, loss, trust, betrayal. Even the musical score is stamped with lingering whisper of time. The title in Chinese is beautiful too: 美国往事, isn't it.

Refactoring of Thinking

blackinkbottle — Sun, 23 Nov 2008 16:46:00 GMT

I have been quiet, rather, I have immersed myself in a lot of reading recently (most of them belong to very random subjects).

There is one related to software, “Pragmatic Thinking & Learning – Refactor Your Wetware” by Andy Hunt (recommended by my colleague ty). It is a nice book, I would recommend it to anyone (particular software developer) who is interested in learning the limitation of our thinking. With the awareness of the limitations, you may improve the important thinking and learning skills, which would naturally lead to productivity and active and efficient discovery of unknown unknown.

Things can work, but not as good as if your thinking had been better. Learning can be fruitful, but not as effective as if we spend time tuning the skill of learning.

The book has a strong dialect towards "software" in general. At the same time it takes many references or pointers from various other fields (healthcare, cognitive science, behavioral psychology, education theory, investment thoery etc) to give different perspectives and combine the ideas from those disparate fields.

It discusses the importance of context, the growth pattern from novice to expert (Dreyfus model: novice, advanced beginner, competent, proficient and expert), the balance (or interplay) between logical (L-mode) and creative (R-mode), the (good or bad) habitual tendency of human mind (cognitive biases, generational affinity, personality tendencies). Given the above backdrop, Andy Hunt prescribed (in the convention of the “Pragmatic Series”) numerical ways and techniques to improve thinking and learning skills (learn deliberately, SMART, SQ3R, learn by teaching, gain experience, embrace error, manage focus, etc)

I like the discussion on context and the repeated assertion that context is the king. This is particularly relevant for the world of software development (from solution artchitecture, to feature design, to library design, to design of data structure and algorithms). Software traditionally emphasizes "encapsulation”. That is to allow nothing but different level of abstractions, or context.

Taking an extended analogy, if every problem (low level or high level) is designated as a context, we essentially perform “context switch” within our brain. Our brain is similar to an OS. The OS should spend more time in crunching the context itself (solving the actual problem); rather in switching the context (the switching latency has to be measured in terms of retrieving and reloading of the entire working memory). When the context is fully loaded, we will decompose the problem into “system calls” which could be handled by the OS.

Therefore, important metrics of the OS are

Latency of context switching (also discussed in this part article)
Clarity and maturity of “system calls” (organized experience)
Stability and reliability of OS (bugs in thinking or even in experience?)

This OS is not open sourced, neither could be virtualized, it belongs to your and me, ourselves. However, it does allow to insert breakpoint for some verification, and allow to perform a core dump for introspection. After all, there is only one rule, there is no substitute for thinking, and there is no substitute for improving our thinking.

Reflexivity suggests

blackinkbottle — Sat, 01 Nov 2008 17:20:00 GMT

Reflexivity suggests feedback loop between "cognitive function" (analogy: sensor) and "manipulative function" (analogy: control). However, in social science or in financial market, the "cognitive function" is flawed due to the imperfect knowledge of participants. As the sensor is not reliable, any attempt for the "manipulative function" to form a negative feedback loop relying on rational expectations (such that market reaches equilibrium postulated by economic theory) will not produce reliable control, not to mention that the control is formed collectively but totally uncoordinated. We may end up with a positive feedback loop, when that happens and when it sustains, we have crisis towards the end of an instable system.

2008.11.02, reading of The New Paradigm for Financial Markets

When the ideology falls

blackinkbottle — Sat, 25 Oct 2008 10:16:00 GMT

When the ideology falls, it crashes. "I made a mistake in presuming that the self-interests of organizations, specifically banks and others, were such as that they were best capable of protecting their own shareholders and their equity in the firms..." (Greenspan, Oct 24 2008) The ideology of free-market capitalism is questioned (or rather sadly, trialed), but who is to question the "greed" of all the contributors? 利令智昏, again.

Google's utility computing

blackinkbottle — Wed, 28 May 2008 15:02:00 GMT

Utility computing, even now still at their first few steps (of thousand miles, such as Amazon's), looks to be one destructive force of the future.

Google has made available their utility computing infrastructure too: Google App Engine
http://code.google.com/appengine/docs/whatisgoogleappengine.html

Kopi Syntax

blackinkbottle — Thu, 08 May 2008 02:24:00 GMT

"Kopi is the next generation multi-purpose service-oriented universal programming langage for rapidly developing software in shortest possible time at cost of fractions of otherwise man-light-year efforts."

Oh...?! Well, in fact Kopi is just to help me and (hopefully) you order Kopi in Kopi shops (Kopi is the blended coffee in Singapore).

The first paragraph is also a "typical" sentence you can find in many bid documents. I am involved in writing one at this moment. Deadline rings yet my colleagues and I find ourselves from time to time doing anything else but working on the docs (sounds familiar?!)

The below resulted from an after-lunch session when the inspirations of writing are "at valleys": we managed to work out the syntax for ordering Kopi on the whiteboard.

	O (Hokkien)	Gao (Hokkien)	Siew3 Tai(Cantonese)	Geh Teng(Hokkien)	C (English)	Kosong(Malay)	Peng(Hokkien)	Halia(Malay)
Water		Less
Coffee		More
Condense Milk	No				No
Evaporated Milk	No				Yes
Sugar			Less	More		No
Ice							Yes
Ginger								Yes

Credit also goes to oc and at.

The rule of thumb is that you can combine any column from the header, as long as there are no two columns having conflicting row entries.

eg Kopi O Gao Siew Tai is coffee black (no milk), thicker (less water, more coffee) and less sugar!
eg Kopi Siew Tai Kosong means "syntax error"!

[Post started on Apr 03, published on May 08, upon return of business trip]

Thoughts on Recruitment and Interviewing of Technical Role

blackinkbottle — Sat, 29 Mar 2008 05:57:00 GMT

Recruitment is an extremely important (if not decisive) factor to the success of a project. This is very true for software project too and some even go to the extend to say that the success or otherwise of a project is determined on the first day the project recruits its full team. The significance could be partially attributed to the unique nature of software related work. I always believe that working with software requires the meticulousness, being analytical towards details and disciplines from the spirit of engineering, yet it also allows the freedom to create without fear of criticism and allows you to give lasting impressions to your audience (ie your user or another software). It is this nature and its implied demanding that makes the recruitment more difficult.

This suggests that a good candidate must possess both traits, analytical problem-solving skill in engineering and capacity to originate creativity and develop useful/practical design. A good candidate has a balanced ratio of mix and intuitiveness of each. Whether there is an magic ratio number is not as important but there must be minimum of each component to allow appreciation of both worlds at the same time - indeed they complement each other.

Interviewing candidates is a very complicated and dynamic process and requires more than just casual thought to make it right. Screening of technical talent for software project requires looking at both hard technical skills (fluency in tools of trade and technologies) and soft technical skills (capability to decompose problem and design). In the case if the role has to lead the project, it requires also the aptitude to communicate effectively to different kinds of people (team members, management, existing customers, prospective customers, marketing/sales forces): technical communication skills.

It is usually easier to determine the level of hard technical skill, by asking the type of black/white technical questions surrounding the topic (programming language, algorithm, technologies). Ideally we should reasonably trust what the candidate tells us on his/her resume or during the interview, but it is typical that the resume is inflated or the candidates tells they are better or more experienced than they actually are due to the incentive of selling. Questions of such could filter out those no hires. You can find better candidate if they respond faster than the average (fluency), or even better one if they can provide additional oversight or consideration of exceptional cases (experience). Again, what is important here is not the final answer, rather the chain of thought which reveals how much the knowledge or best practices has been burned and hard wired into their thinking and those have become just too intuitive to allow pause for retrieval (retrieval has long latency if they only just try to recall from the preparation before the interview). Only by gaining such fluency, the bottleneck of expressing any design ideas could be eliminated and one could concentrate on the actual design work rather than spend time in translating the ideas into physical bytes.

It is much more difficult to determine the level of soft technical skill or the actual skill of designing software. Design is partially scientific in terms of capability of composing the big picture with bits and pieces, but also is largely a reflection of personal styles and values. Design is about trade-off and people decides trade-off, consciously or unconsciously based on their personal beliefs. It could also relate to self-knowledge, how we think we appear to others, how we think others evaluate that appearance probably against some benchmark. That results in shame or pride we feel and we may inevitably align with that benchmark. That could explain why frequently the latest technology, the paper design in the patterns book, the best practices from the industry white paper become the output to align with rather than the input to the thought process of software design. Finding the perfect candidate is certainly very ideal situation and luck usually plays an important factor too. It is also true that the maturity could be developed but the potential is the key. That said, measuring the competence of designing software or the future potential has to go through some metrics.

One metric is to see how a candidate possess an intuitive capability to handle multiple levels of abstraction (from design of software/system, to design of sequence interaction, to design of class in separating of concerns, to design of eg longest common subsequence algorithm, and down to eg pointer arithmetic) at the same time. He/she should be able to switch among those different levels of details very very quickly. Any loss in chain of thought (remember, we are talking about the thought, not the answer) probably means that some people do not get it or just can't do it; any slowdown in chain of thought is an indication for lack of fluency.

Another metric is how good understanding the candidate has about "trade-off". This could be partly revealed in their attitudes towards architecture. Many people talk about architecture without truly appreciating its significance. In fact most of the time it is because there is common belief or tendency to consider architecture or software architect more prestige than the final implementation or those who implements (software engineers, programmers, coders). The exact thinking that the latter, who designs an implementation is not doing the creative design work of the same magnitude, reveals the lack of experience in designing software, not to mention the design of architecture. It is just at a different abstraction level. In my opinion, I always prefer someone who is humble, down-to-earth in talking about software architecture and has mastered the tools of trade for conveying and communicating the idea of designs. They are usually those who along the bumpy roads, learned and started to appreciate the series of "trade-off" in designing software.

The final aspect to watch is the technical communication skill. This is important for every team member but is extremely important for whoever need to lead the team or act as the interface to the outside world. The candidate must be able to explain things well and with passion. During the interview, there should be no drifting in explanations or avoidance of hard questions. If ever provoked (if you have chance for such a question), the candidate should not stand on the fence; but also on the other hand never never argue for the sake of arguing. How the candidate can provide a convincing middle ground to close the topic reveals the capacity of influencing others.

Talking about technical recruitment cannot avoid the topic of career path. Many people has disbelief in the technical career and they cannot be blamed. "Managerial career is the only option" - this is in fact quite common belief in Singapore, also in Asia. This belief is an indicator of several opinions of (possible) facts:

- people consider managerial position more prestige than technical position

- society tends to reward managerial position more materially than technical position eg there might be simply perceived gap in pay cheques

- success tends to be attributed to managerial position and failure frequently attributed to technical position

- managerial positions are cycled faster (promoted) than technical positions

- managerial positions suffer from soft stress and technical positions suffer from hard stress

- incompetent management could be tolerated at times; technical decisions could be sacrificed when required (remember NASA Challenger)

- managerial positions are profit center and technical positions are cost center

- etc etc

This belief also acts as self-fulfilling prophecy. Pay/rewards drives more supply, the abundance of supply (for manager career) and lack of consistent high standard would lead to more incompetent management - but again they could be tolerated - this only enforces the belief. Given the respective reward/risk ratio, it is not difficult to see the statistical distribution of choices. The above is the collective value of the society, it is hard to change. To attract technical talent, the employer has to do the action. There is only one way to stand out among the crowd - to build and enhance the company culture of valuing "value generation", and separate the wheat from the chaff.

Sometimes there is also misconception that management is just easy and all about watching over the shoulders, which sounds like the "reward" for completing the rat race in the junior roles. Wrong! That happens in many industries and in many fields, a reality of the life. But the motive being completing the rat race just sounds dubious for the devotion of contribution. Taking up managerial role is definitely a natural extension of responsibility: a soldier who does not want to become a general is not a good soldier but a solider who just wants to become a general so could no longer fight the war is definitely not a good soldier.

Another caveat I would like to bring up is, good candidates are hard to come by, luck plays its part too. But do not limit your hire/no hire selection only to the current batch of potential candidates you may have at hand. It's sometimes tempting to lowering the standard if other attributes of the candidate are likable (genuinely good people are many but good people does not mean they are suitable for the given role), or if there is pressing needs from the project. Give you time to search harder, if the role is important to the project and if the project is important to you.

I am advocating elitism? No. Some are lucky and born with talent, most are not and have to earn/nurture it with hard working. What I said is that I do not advocate equal outcome, I advocate equal opportunity that everyone can be or become one talent.

Another mind map on Integration Topology (Message Broker, Message Bus etc)

blackinkbottle — Wed, 03 Oct 2007 02:23:00 GMT

The earlier post of mind map focused on WCF. The below is more on integration topology, with more contents covering Java related technologies.

Integration Topologies

Message Broker (Hub-and-Spoke)
- Technology Stacks
  - Client Library Implementation
  - Communication Protocol
  - Broken Implementation
- ActiveMQ
  - Communication Protocol
    - OpenWirev1
    - OpenWirev2
    - STOMP
    - REST
    - WS-Notification
      - ServiceMix
    - XMPP
      - Jabber
      - Client Lib - Spark
    - AMQP (incubation)
    - RSS, Atom
  - Container
    - Axis Support
    - Geronimo
    - J2EE
    - JBoss Integration
    - JCA Container
    - JNDI Support
    - Resource Adapter
    - Spring Support
    - Sun JNDI
    - Tomcat
    - WebLogic Integration
  - Client Lib
    - ActiveMQ C++ Clients
    - Ajax
    - C Integration
    - CMS
    - NMS
    - Perl
    - PHP
    - Pike
    - Python
    - Ruby
  - Coding
    - ConnectionFactory
      - Connection
        Session
        Destination <= createTopic
        Destination <= createQueue
        MessageProducer <= createProducer
        setDeliveryMode
        send(Message)
        IMessage <= createMessage
        MessageConsumer <= createConsumer
        setMessageListener
        OnMessage(Message)
Message Bus
- ServiceMix
  - More details...
- JBI
  - a pluggable architecture for a container that hosts service producer and consumer components
  - Services connect to the container
    via binding components (BC)
  - or hosted inside the container
    as part of a service engine (SE)
  - Message Delivery
    - NMR Normalized Message Router
    - MEP Message Exchange Patterns
      - In Only
        A standard one-way messaging exchange where the consumer sends a message to the provider that provides only a status response
      - Robust In Only
        This pattern is for reliable one-way message exchanges. The consumer initiates with a message to which the provider responds with status. If the response is a status, the exchange is complete, but if the response is a fault, the consumer must respond with a status.
      - In Out
        A standard two-way message exchange where the consumer initiates with a message, the provider responds with a message or fault and the consumer responds with a status.
      - In Optional-Out
        A standard two-way message exchange where the providers response is optional.
  - Lifecycle
    - JMX
      - installation, deployment, monitoring and lifecycle concerns amongst BCs and SEs
  - More Info on JBI...
    - JBI is an architecture for integration systems specifying plug-in components that interoperate by exchanging messages, rather than by interacting directly. This decoupling increases flexibility because each component needs to know how to interact with the JBI bus only and not with n number of other components. JBI components provide services, consume services, or sometimes both. There are two types of components: Service Engines (SE) and Binding Components (BC). The SEs provide business logic and transformation services. The BCs provide connectivity for applications that are external to the JBI. The separation of business and processing logic from communication logic makes the implementation of components much less complex.
      
      The mediated message exchange between components is provided by the Normalized Message Router (NMR). The NMR routes normalized messages between service providers and consumers. A normalized message consists of two parts: the message content (payload) and the message metadata. The message metadata contains information such as security information, that can affect the processing of the message as it routes through the JBI. Messages flowing into the JBI, via binding components, are translated into a normalized (neutral) format, then routed to their destination. Prior to final delivery the normalized message is translated into the appropriate format for the recipient. A message can be routed through several JBI components depending on what processing is needed.
- Enterprise Service Bus
  - Implementation Camp
    - multiple protocol
    - purely on SOAP/HTTP
    Marketing Attributes
    - distributed processing
    - intelligence routing
      - visibility - into messages for content-based routing
    - security
    - dynamic data transformation
    - business process orchestration
    - reliability
    - transactionality
    - Event Driven Architecture
      - This is an asynchronous operation as the recipient applications
        may receive or pick-up their messages at a later time, whereas
        the SOA messaging model is typically synchronous in nature.
    - quality of service (QoS) features
      - message persistence
      - guaranteed delivery
      - failure handling
      - transaction support
    - comparison
      - EDA
        decoupled, scalability and reliabilty required for high transaction production environments
      - SOA
        synchronous request/response communication (usually)
    Infrastructure Characteristic
    - distributed - to remove geographical constraints
    - message-based - to promote loose coupling
    - open standards-based - to preserve investment and encourage partipicipation
    - reliable - to meet the requirements of mission-critical business operations
    Design Requirement
    - Standard Based
      - No product API lock in
        JIB
        POJO
    - Flexible
      - Deployment
        standalone
        embedded in an application component
        as part of the services supported by an application server
      - Component Reuse
      - Scripting Language
      - Embedded Rule Engine
    - Reliability
      - network outages
      - system failure
    - Breadth of Connectivity
      - two-way reliable web service
      - MoM (Message Oriented Middleware)
    Enterprise Need
    - Disparate Application
    - Disparate Platform
    - Disparate Business Process
    - Non compatible data format
    - Non compatible communication protocol
    - Need: Enterprise Integration
      - Past Solutions
        Enterprise Application Integration
        Service Oriented Architecture
        Web Services (already a past ???)
      - Evaluation of Past Solutions
        Vendor Solution
        high cost, lock in
        Custom Solution
        high maintenance, host cost
    The ESB addresses the disadvantages of existing solutions by
    creating a standard infrastructure for integration.
    Point-to-point solutions, where each of n components requires n-1
    interfaces for full communication, are replaced by a bus solution
    where each component requires a single interface to the bus for
    global communication
    Debate: is it relevant?
    - http://blogs.msdn.com/mohammadakif/archive/2007/01/23/microsoft-s-enterprise-services-bus-esb-strategy-part-4-4.aspx
    Microsoft's ESB Offering
    - Application Platform
      - WIndows Server
      - .NET Framework 3.0
      - Core: BizTalk Server
    - Advertised Features
      - Industry Leading Web Services Support
      - Intelligent Routing
      - Message Transformation
      - Itinerary Processing
      - Legacy and LOB Application Adaptation
      - Service Orchestration
      - Metadata Lookup
      - Exception Management
      - Distributed Deployment
      - Centralized Management
      - Business Rule Engine
      - Business Activity Monitoring
Web Service
- POJO (Plain Old Java Object) Exposed as Web Service
  - Spring Based
- Axis2
  - Transport Receiver - transport related processing
  - Dispatching - find service and operation
    - AddressingBasedDispatcher
    - RequestURIBasedDispatcher
    - SOAPActionBasedDispatcher
    - SOAPMessageBodyBasedDispatcher
  - Message Receiver - last handler of the chain
  - Configuration
    - ServiceGroup
      - Service
        Operation
        Message
        Service Scope
        Request Scope
        SOAP Session Scope
        Service Group Id
        Transport Session Scope
        Cookie
        Application Scope
      - Data shared among services
      - Session maintained using context
        Login
        Logout
- JAX-WS Java API for XML Web Services
- JAXB Java Architecture for XML Binding
- Microsoft WSE, also WCF
Publish/Subscribe

Of course, this is collected facts or believed facts. There is no judgement or recommendation. You have to do your own research, hands on verification before believing. As usual : )

A mind map for Windows Communication Foundation

blackinkbottle — Wed, 03 Oct 2007 01:50:00 GMT

I spent some time in space walking various communication technologies for comparisons. They aren't really new, not so "on the center of the cutting edge" (I know you have seen this phrase!)

The below is generated from FreeMind. By the way, FreeMind is a terrific tool for brainstorming or knowledge consolidation, or even sales demonstration. I would love to see any Flickred mind map sharing site, or perhaps in future Google Docs?

Hope the below is useful for situation requiring at-a-glance or quick recall.

WCF

Endpoint (ABC)
- Address
- Binding
  - Transport Protocol
    - HTTP
    - TCP
    - Named Pipe
    - MSMQ
    - P2P
  - Message Encoding
    - SOAP, Text
    - Binary
    - MTOM (Message Transport Optimization Mechanism) - optimised for large binary payload
  - Communication Pattern
    - Datagram (fire and forget)
  - Reliability
    - Reliable Messaging
  - Security
    - Transport Level
    - Message Level
  - Transaction Propagation
    - Atomic Transactions
  - Interoperability
    - WS-*
    - raw MSMQ message
    - basic Web Service
- Contract
  - Data Contract
  - Message Contract
  - Fault Contract
Comm Layer
- Intra-Process
- Inter-Process, same machine
- Inter-Process, across wire
Hosting
- IIS
  - HTTP transport only
- Self Hosting
  - ServiceHost(typeof(ABCService), uriBaseAddress) (per transport protocol)
    - ServiceHost.Open
    - Service.Close
  - a physical process to host it
    - process boundary
    - machine boundary
Communication over Wire illustrated
- Client
  - Proxy
    - Channel(s) pipe
      from client -> service pre-processing
      from service -> client post-processing
      - Transport Channel
        ... Wire ...
        Transport Channel
        Channel(s) pipe
        from client -> service pre-processing
        from service -> client post-processing
        Dispatcher
        Business Endpoint
        User Code
        Message pre-decoding post-encoding
        bindary
        text
        MTOM
        set security context
        propagate client transaction
        manage reliable session
        pre-decryption post-encryption
      - Message encoding pre-encoding post-decoding
        bindary
        text
        MTOM
      - set security context
      - propagate client transaction
      - manage reliable session
      - pre-encryption post-encryption
Scenario

Intermediary Router
Federation
List-based Publish Subscribe
Trusted Facade Service
Discovery
Peer Channel Custom Peer Resolver
REST and POX (Plain Old Xml)

Readings and Questions on Software Architecture and Software Architect

blackinkbottle — Tue, 21 Aug 2007 15:53:00 GMT

Should software architecture be owned and how to measure the output? What are responsiblities and liabilities of software architect?

I really like some articles from Skyscrapr because it provides in my opinion a good amount of quality, proper and down-to-earth (humble and not too much space walking) education material to what software architecture (responsibility and output) should be and what an architect (owner) should be concerned with.

In any sizeable and sustainabale human endeavour, it is impossible to imagine a teamforce without proper, formalized roles and responsibilities and also ownership of output. Yet, that could happen and happened for software. Software architecture is a grey area in many enterprise projects, software architect is a grey role that is so ambigously defined that whether he should code or not may become debate of years without clear answers.

In the article, What is an Architect brings the question upfront to the table and proposes the common terms the industry should agree upon: Solutions Architect, Infrastructure Architect, Enterprsie Architect. That is great clarifications - the reminding of roles and responsibilties would ensure the consistency and measurable of output!

The question is part of the original blog article from Software Architecture: Past, Present and Future which outlines the evolving of need of software architecture and software architect.

There are some other good articles, such as Mentoring: A Natural Element of Architectural Leadership. We also have Are We Engineers or Craftspeople?

I always consider raising the right question is much more important than finding the correct answer. I would also applaude the sections of "Critical-Thinking Questions" in some of the articles (eg, Are We Engineers or Craftspeople?, The Role of Architecture in Business Analysis): they are so refreshing! The huge amount of articles and resources avaiable today, and avaiablility of google wikipedia at finger tips frequently mislead our mind into vicious cycles of instance-noodle trap - articles are read and forgotten, such sections reminded the need of asking the right questions and thinking out (not reading out or google out) answers. Great! I will re-read starred articles another 2-weeks then 4-weeks to ensure I still retain what I am enlightened with.

In the end, all the above cannot escape the center piece "communication and communication": architecture is a design output to communicate after all, to influence the decision, to justify the argument, to navigate through all the different waters. When we convinced others, we built the skyscrapr!

Value of C++, Or Do I Live in Denial?

blackinkbottle — Sun, 22 Jul 2007 13:10:00 GMT

I chatted with a friend yesterday (who worked mostly on C#/PHP and even Ruby/Rail for web applications) about the value of C++ language, well, you can probably imagine the content of the discussion and there would be some debate.

I recalled another discussion with my colleague sb sometime at the beginning of this year, I retrieved the correspondence, re-read it again and think that my belief is still valid.

That discussion originated from two articles on technical interviews with Bjarne Stroustrup, available at the first part and the second part. Bjarne gave two interviews to Technology Review (published by the MIT) where he talks about what's good and bad with C++ and software development at large. (by the way, these two are very good articles, even some of the comments.)

== Start of Discussion

Me:
Certainly people in the industry or in the academic are all comtemplating this.

It is tempting to declare something is perfect and this is seldomly true. I prefer to compare software development to composing music or painting: there is one way of conveying things and it is bound to have some other different means. Music and paint are aesthetic outputs. Architectures, designs, maintabilities of software development are all such aesthetic goals, as if you searching for listening to "good" music, or viewing the "grasping" paint - although they all relative to the ears and the eyes.

But software development requires one more thing than the "ways" to express those aesthetic styles we all search so hard. It requires correctness - nothing comes before it, every other thing is secondary.

Also from the above interview, "The purpose of a programming language is to help build good systems, where "good" can be defined in many ways. My brief definition is, correct, maintainable, and adequately fast." I cannot agree more. "fast enough" is aesthetic goal too at first, though sometimes it is defined as part of correctness for eg real-time systems. Correctness is the one and the foremost something the software is created for either economical or scientific reasons.

Saying this does not say that correctness would overshadow the value of architectures, designs, maintabilities, responsiveness etc. Surprising, when striving to achieve the correctness which definitely requires more careful thinking, a system, by chance or as a consequence, that amount of thorough thinking naturally may lead to other good outputs in architecture, design, or maintability.

I would say "correct systems are more likely to be maitainabale, and more likely to be beautifull too when looking into the architecture or design"

So much of my mumbling...of course, there is yet another "Bjarne Stroustrup Interview"
http://www.erenkrantz.com/Humor/FakeIEEEStroustrupInterview.shtml - well, you got what I mean. *)

His reply:
Haha this is very funny.

Seriously, I think it's good to consider the software as a piece of art, as if one was looking at the bunch of equation written by Einstein and saying "waaa it's beautiful...". It is indeed.

But, personally I think we are nowhere close to that yet. I think the metaphor with the architect was perfect: he has the freedom to design masterpieces, but yet, to do that, he has to use a set of perfectly well-know and well-defined tools. He knows that to achieve task A, he needs to go through task B, using tool C and D. Or just like the physician. I would like things to be that simple in computer science. But we haven't yet reached that state.

My reply:
Yes. My thinking is along the same line. I emphasize correctness at the most important thing to consider. By itself correctness is not strictly an "aesthetic" goal (indirectly yes), and I think it comes before any other goals more related to aesthetic aspects such as architectures, designs, maintabilities, responsiveness etc. Somehow the industry leaves the impression that architecture is the foremost important - I disagree to an certain extend.

What software development differs from painting and composing music is the correctness - its mathematical spirit. Correctness is a complex task - but it is a "simple" goal to achieve, and "could" be done "simply" in more "systematic" way, with more "objective" evaluation. The striving for correctness could be thought of equivalence to countless amount of pratice effort required by a good physician, musician and painter to be fluent. The aesthetic outputs such as architecuture etc are merely the natural extension of searching for correctness, when the knowledge becomes wisdom.

His reply:
The thing is with C++, so many people have no notion of that "correctness". They attempt to write C++ without knowing how it should actually look like. They've never read the STL, never read Boost library, and think "oh it's like C", but it's so different.

The freedom given in this language allows you to define any problem in mathematical terms. There is a certain way of doing it, using a certain syntax. People should not try to write C++ when they don't know what it should look like.

The fact that it was compatible with C brought the idea widely accepted that anything can be written in C++ like C. This is doing more bad than good to C++. Anyway given the wide range of specific high level programing languages available nowaday, I think C++ will finally be found only in the area where it is really good at: complexity/performance of data processing applications.

== End of Discussion

Coming back to C++ language itself, I I cannot agree more with the opinion from one author called JF, saying (titled: C++ Design Philosophy, comments to the first part of the interview)

I think it is hugely important to go back and read exactly just what the design philosophy behind C++ was (and still is). Most of the writting (especially those against C++) seems to be demands for more features(eg garbage collection, pointer management, threading support etc). But when you carefully review the thinking behind C++ (as I have done -by the way I have developed applications in Fortran/C/C++:numerial/graphics analysis and realtime telco apps and there is no other language that could have done the job-maybe at the time.) BS and the standard's committee are keenly aware of all these 'shortcomings'(if you want to call it that) of the language . A fundamental aspect of C++ is that it should support multi-paradigm programming -that in itself is a tall demand. But more improtantly, C++ should allow you to be as close to the machine as possible but 'no closer' while retaining 'expressivity'. In the end, there is no substitute for attention to detail and that is the programmer's job. If the machine is kind enough to give you memory when you needed it (and don't force you to give it back) then just make sure you return the favour. I believe we do and will need a language that allows you to do what you want. Incrementally though, features are being added to C++ to resolve exactly some of these issues. Practically, there is no reason to use old fashion/native arrays over vectors. It is my believe that some day threading support and memory management will be part of standard C++ once an efficient implemetation is found.

I would point anyone who is now still interested in C++ to the above two articles, read more about C++ and its philosophies. Maybe only do it only for historical interests (because I have to admit, not many current jobs require you to write C++), the efforts of stepping into un-comfort zone would pay off. When you force yourself to do something that you don’t understand and don't see immeidate pragamtic applicability, the initial results are always messy - but that experience is always original, extends your horizon and gives you a lot more visibility into the complication of the software world. For starting in C++, I would always recommend by starting directly in STL (eg some of my old posts in this forum here, here, here, here, and here, forgive sometimes my nugging).

Google Desktop for Linux Desktop

blackinkbottle — Thu, 05 Jul 2007 04:17:00 GMT

Nearly 3 years after Google Desktop making a debut on Windows, it's time for its presence on Linux Desktop. It's available at http://desktop.google.com/linux/, requiring glibc 2.3.2+, gtk+ 2.2.0+.

Some brief look is also available here.

Henri-Georges Clouzot, Picasso, Painting and Software

blackinkbottle — Sat, 30 Jun 2007 09:49:00 GMT

There is nothing more refreshing than to watch an interesting film that combines the art of film, the art of suspence and the art of painting together. I was drawn to the exhibition from Singapore Tyler Print Institute, on "Pablo Picasso: The Vollard Suite (1930 - 1937)", firstly because of Henri-Georges Clouzot. His two other "Les Diaboliques (1955)" and "Wages of Fear / Le Salaire de la peur (1953)" had left me with lasting impressions and I was eager to see how he saw the Mystery of Picasso (Le Mystere Picasso (1956)).

Clouzot's intention is convincing: if we could see the entire process of painting in temporary dimension, we might be able to peek a glance into the mind of this great artist. The film set was extremely simple, Picasso sat on one side of his canvas which had a special transparency characteristic that could absorb and let ink bleed to the other side where the film camera was placed, capturing every single stroke or brush, color and layer that was applied, the painting (although effectively being a mirror paint) became alive!

There are about 10-20 pieces of different paintings that are played back from empty canvas to completion. Nearly for every artwork, the beginnging is just seemly arbitrary strokes, but subsequent some pattern emerges gradually, becomes clearer, then sometimes gets dissolved, transformed and morphed into something else - the changes are the mirror of the creative process - and this is also exactly the art of suspence of film is all about too. There is hardly any hesitation of where to put the next stroke, where to recoincile initially unrelated pieces. During the climax of the beach scene (as every suspence film should have), the canvas has experienced transformation of countless times, it is almost about several tens of layers of painting in one, as Picassso effortlessly moved his brush to show off, a canvas, empty or otherwise full of arbitrary color and layers, to him is just the same, as he mutters "its ruined, I have ruined the painting and yet at the same time, it is improving", I suddenly understand that "the most rebellious is the definitive interpretation".

About software. I relate the seemly arbitrary stroke to realization of software design; I relate repeated changes and scene transformation to refactoring. You can always change your mind during the latter, but make sure the mental design is already well shaped in your mind. To achieve that, you can hardly rely on the paper design in the patterns book, in the industry white paper, from the best practices, from the software factory: they should merely be the input to your thought process, rather than to be the output to align with.

Whether software programming is an art or a science has been debated for decades: I don't know the aswer either. But I strongly feel it is a gifted area that shares the best part of both worlds, it requires the metriculousness, analytical towards details and disciplines from the spirit of engineering, yet it possesses the freedom to create without fear of criticism and allows you to give lasting impressions to your audience (ie your user). Don't they say (Knuth): "let us change our traditional attitude to the construction of programs. Instead of imagining that our main task is to instruct a computer what to do, let us concentrate rather on explaining to human beings what we want a computer to do." I see this shares the most resembling trait with painting and composing music.

Go and watch the exhibition if you can, it lasts till 7 July, check out the details at Singapore Tyler Print Institute.