My update covered the two paragraphs that immediately followed the last line of sample code. The original text of these paragraphs was as follows:

Now there are two issues that should be emphasized about the above examples. First, the copy semantics of structs exclude any array members, so the member a will not be initialized or modified in any of the examples except the first two. Second, the last example is very inefficient since, disregarding any possible optimizations, gee will first be copied to the parameter, then to the result, then to golly.

Though I am now sure that the code I was seeing is legal; given the inefficiencies of copying structures and the large number of structs being set, I believe the originators of the application I’m dealing with would have significantly improved performance by using a pass-by-reference (pointer) strategy. While I have worked with structures in countless programs, I tend not to do direct struct-to-struct assignments, preferring to use memcpy. The latter works whether the struct contains member arrays or not, so treatment can be consistent throughout and across applications.

While I am grateful for Derek Jones response, and I accept his interpretation as authoritative with respect to the intent of the standard, my own experience and articles/advice I’ve seen from other developers leads me to believe that at least some embedded compiler vendors shared my original interpretation; so I recommend that you tread carefully if you decide to use struct-to-struct copy with array members. Run a few quick tests to make sure the implementation performs as you expect it to.

As I side-note, I was very impressed with the Coccinelle tool that Mr. Jones mentioned. I am quite sure I will be able to make use of it; and perhaps I’ll give it a review here then. In the meantime, it looks amazingly useful for anyone who is maintaining or updating a large code base, either for use in review or bulk-update.

All named members of a structure are required to be copied by a
conforming implementation, including arrays.
The only places where implementations may vary is in the handling
of any padding between named members, they are not required to
copy this.
See sentence 1298.

I can see how you might connect 6.5.16.1 and footnote 42 to think
that members having an array type would not be copied. The reason
that arrays cannot be copied is that they degenerate to pointers to
their first element, loosing all ‘arrayness’ information in the process.

Unless the struct is very small it is much more efficient to pass its
address. Of course this usage requires that the called function does
not modify the contents of the struct.

As s1298 struct assignment can be more efficient than using memcpy (gcc
does do fancy things to figure out alignment of the objects passed to
memcpy). Member by member copy can be more efficient for low numbers
of members because it has no loop overhead.

If you are making lots of very similar changes to C source you might be
interested in using Coccinelle:
http://shape-of-code.coding-guidelines.com/2009/01/08/semantic-pattern-matching-coccinelle/

You are welcome to copy-and-paste this to the comment section of your blog.

–
Derek M. Jones tel: +44 (0) 1252 520 667
Knowledge Software Ltd mailto:derek@knosof.co.uk
Source code analysis http://www.knosof.co.uk

Thank you Mr. Jones for your kind and prompt response.

I do not know the standards well, so could you direct me to where in they state that the copy semantics of structs exclude any array members. This seems inherently wrong not to copy all member of the structure. It could also take a alot more instructions, depending on how many array intermixed in the structure, to do a selective copy than just copy the whole structure.

As I said, these are my interpretations of the standard. In reviewing the materials to respond to your question, I found that I had misread one section, and failed to consider the technical corrigenda. I also dug a bit deeper by looking at “The New C Standard”, by Derek M. Jones. I now believe that a more correct interpretation is that the copying of array members in structs is implementation optional.

The rule in question is not explicitly stated anywhere that I can find, and I am not entirely confident that my interpretation is correct.
Nonetheless here are the areas that lead me to this conclusion:

• Section 6.5.16.1 Makes no provision for an assignment to an array. (i.e. direct array-to-array copy is not supported).
• Footnote 42 to Section 6.2.6.1 par 6 indicates that a structure assignment may be copied via a memcpy or via element-by-element copy. If the latter is used, I would expect it to be via the rules of the previously mentioned section.
• Section 6.7.2.1, was heavily ammended by TC2. According to the new paragraph 22, which is discussing flexible array members, it indicates that the portions of the flexible array member which reside within the structure size may either be copied or overwritten by arbitrary data. While this is a specific case, the treatment seems consistent with my interpretation.

By the way, inefficient or not, the compiler implementation I am dealing with is performing the member-by-member copy of structs.

The purpose of the article was to share a powerful technique for manual static analysis. Actually I work with a number of automated analysis tools. I am very familiar with PolySpace. I have trained people in its use, and I have sold it into client sites. It was one of the static analyzers used on this code, and it did not and could not find the issue; and we could not find a modeling technique that would permit us to find it. The capabilities of the tool are a bit stronger now than they were then, but even in the best case PolySpace would have required extensive setup, code annotation, and modeling. PolySpace also had some issues in how it dealt with bitfield values and bitwise operators in C. I don’t believe that any automated analysis tool in existence could have done this job. The viable options were formal methods, simulation, and manual analysis. Formal methods suffers from fidelity issues, and simulation was done in parallel by three of the original developers using MATLAB. The simulation team also found the issue at about the same time the analysis did, but they had a two or three day head-start.

It is not true that PolySpace creates all possible test cases. It uses abstract interpretation to mathematically try all possible values for a variable value at every point in the program. In many cases it also tries values that are not possible, and values that are not possible in combination. This makes it valuable for detecting semantic errors, but without extensive modeling it generates many false positives and while it detects where exceptions might potentially be thrown; it is fairly useless for determining correct decision making in code not specifically written to take advantage of its analysis strengths. Setting up for a PolySpace analysis of a substantial program can take a great deal of time; and if I recall correctly, the run time of the PolySpace analysis on that application was actually longer than the time I took for my manual analysis; and we had to run several times as we modeled to reduce false positives. We had the fastest PolySpace server configuration available at the time.

As I said, our professional practices are in vastly different arenas, and I would expect our perspectives to be very different. I also must apologize that I mixed some of what was offered in the presentation with items from the whitepaper; and I suspect that contributed to some misunderstandings on my part. I did not have the benefit of the presentation slides to re-visit some of the presentation topics, so I used the whitepaper as a poor substitute.

It appears that at least part of our disagreement is due to terminology. I was reacting in part to terminology I heard during your presentation; some of which may have been raised in addressing audience questions. Your papers do a good job of clarifying your position on some of these issues. For example, Part 1 of your SIP briefing states: “SIP is not intended to replace existing enterprise architecture methodologies. It is intended to provide a meta-methodology on top of them.” If that was stated in the meeting, I missed it.

The “business requirements” statement was again, in response to what I recall being said in the presentation. You opened the presentation with a statement about the contribution of moving requirements to the cost of projects. This lead me to believe that you were in some way addressing that issue. The term “business requirement” was used at least a few times, apparently in reference to the business functions; and this lead to some confusion.

As to my assertion about multiple database systems, this answered a response you gave to an audience question. You stated that the various subsystems could be contracted to different vendors, and that if they shared a common underlying database system, that would be coincidental. I also spoke with at least a few audience members who had trouble with that idea; as well as each sub-project having to originate duplicative “plumbing code”. These were probably the two biggest objections to the whole concept. If your claim is that EVERY partition, can and/or should be outsourced to a unique development group, that’s where I think the method falls down.

As to simplification, I don’t know what common practice is, but I don’t consider it to be only with code. Design simplification typically yields much greater benefit than code simplification. I spend a great deal of effort ensuring that the architecture is as simple as I can get it. I will suggest again that this has been a best practice for decades. This is exemplified by the following passage from “The Practical Guide to Structured Systems Design, 2nd ed.”, 1988 (Mielir Page-Jones): “Reducing the coupling between modules means, in effect, reducing the complexity of the connection(s) between the modules as much as possible. To paraphrase Einstein, we seek to make such connections as simple as the application will allow, and no simpler.” In the chapter on cohesion, the book goes on to say: “Cohesion is the measure of the strength of functional relatedness of elements within a module.”; similar information about reducing design complexity has been present in every design book I’ve ever bothered to look at. Unfortunately, I suspect that your perspective may be correct, that design level simplification may not be done as it should be in general practice.

I disagree with your assessment that there is a correlation between re-use and complexity. My experience indicates that the biggest barrier to re-use is the lack of effective repositories and indexing tools. Methods such as Shlaer-Mellor and MDA encourage coarse code re-use at the domain (architectural element) level; and that has seen success in some companies, but few are willing to share details of their gains. On the other hand, I think code sharing among cooperating subsystems is very common practice, and tends to lead to better designs. It sounds as if we are in agreement on this; but my argument was that your approach seems to discount the potential value.

As you’ve made your claims explicit, I’d like to address them directly;

• “Synergistic partitioning is based on the mathematics of partitions. No other IT methodology is based on a mathematical foundation.”

  ALL of IT is based on mathematical foundations, from relational database theory to language semantics; and there have been other numerous attempts to apply a mathematical approach to software development (for example, see this 1998 paper). Still, we tend to take an informal approach to all levels of software development.

  In safety-critical domains there is a well known methodology to verification, called “Formal Methods” by which you can prove many of the operational and safety aspects of software mathematically with lower costs and greater efficacy than testing; yet organizations don’t do it because they perceive the formalism and discipline required as a productivity hindrance, rather seeing the benefit of reduced testing. I think that development methodologies tend to be informal because formal algorithmic approaches don’t get adopted. The advantage that your approach appears to have is that it yields some reasonably good partitions with relatively small effort. This improves the chance of adoption.

• “Synergistic partitioning drives a unique solution. For a given universe of functionality, there is only one possible solution. No other IT methodology has a unique solution.”

  I am inclined to accept this claim at face value, though I suspect that differences may arise due to how and to what level of detail the business functions are elaborated; and perhaps also due to assumptions and pre-conceptions about the architecture.

• “The resulting partition is the simplest possible partition. There is no other approach that can discover a better configuration.”

  It may be unique, and functional cohesion and communications coupling are certainly large factors; but I am still skeptical that this unique solution is necessarily an optimal one. I don’t disagree with the partitioning as a useful tool, but I’m not as certain that for the universe of all possible projects, the partitions provided are neccessarily the best divisions, or that they can be always farmed out as stand-alone projects. On the other hand, they are almost certainly excellent starting points.

• “The resulting reduction in complexity can be measured and translated into actual business dollars.”

  I agree that cost variability is probably reduced; only because I’ve seen some absolutely attrocious designs. That the reduction in complexity can be translated to actual business dollars is dependent on knowing how the alternative approach will address complexity. In terms of outsourced projects, I think that your approach, applied by the customer, would reduce opportunism by vendors; even if the entire project was still given to a single vendor.

Since my initial post, I’ve had a bit more time to review your papers. Your SIP(tm) approach certainly has some interesting and useful properties. I will be on the lookout for opportunities to put it to the test.

There are some aspects about your approach that I really do like. As I stated in my post, there have been attempts to quantify complexity, particularly as an aspect of difficulty. I believe your quantification method does provide a good measure. Furthermore I believe that an algorithmic and deterministic approach to partitioning may be of benefit, particularly when pushed up to the business analysis level.

It appears to me that your method may lend itself to automation tools which can simplify the production of the top level architecture. I can also imagine that within each partition a number of partition functions could be elaborated as a decomposition excercise; and that this would permit recursive decomposition of a more complete architecture.

Best of Luck! I look forward to seeing what the future has in store for your methodology.

– Max

You say I have outlined a new approach to creating software architecture. I don’t think this is right. I have outlined a new approach for partitioning projects so their complexity is minimized, thus removing the main barrier to creating good software architectures. Creating a software architecture is something that occurs much later. I would agree, though, that the process partitioning that I recommend puts some constraints on the eventual software architectures that can be created.

You say that the solutions I propose are analogous but inferior to solutions that already exist. I disagree. I haven’t seen anybody proposing system simplification through partitioning of functionality through use of synergy analysis. So we can discuss whether my methods are better or worse than others, but they are clearly unique. Here are four claims I make for my methodology that I haven’t heard anybody else make (and I don’t believe anybody else can make):
- Synergistic partitioning is based on the mathematics of partitions. No other IT methodology is based on a mathematical foundation.
- Synergistic partitioning drives a unique solution. For a given universe of functionality, there is only one possible solution. No other IT methodology has a unique solution.
- The resulting partition is the simplest possible partition. There is no other approach that can discover a better configuration.
- The resulting reduction in complexity can be measured and translated into actual business dollars.

You say my approach forces an organization to more closely analyze business requirements. Actually, my approach doesn’t deal with business requirements at all. Only business functions. In other words, I look at what the system does, not how it does it or what specific requirements it must satisfy in doing what it does. It isn’t that I don’t think business requirements are important, just that they come much later, after we have partitioned the business functionality.

You say I don’t decompose far enough. I disagree. I decompose exactly as far as necessary to identify the atomic business functions. And then I stop. I don’t care how they will be implemented, either at the business architecture or technical architectural level. Not that these things aren’t important. Just that they are a different problem than what I am addressing. And just to be sure we are on the same page, my problem is Complexity. I am assuming that once you have addressed complexity, the other problems (like business requirements) are much easier to address.

You say that a problem with my example, inter-library loans, is that we could end up with multiple database systems and that this will “increase operational and support costs for each of the using libraries.” I disagree. Not only does the system I proposed have no need for multiple databases, it has no need for ANY databases. The inter-library coordinator can do its work on a simple spreadsheet or file system. The Borrower system doesn’t need to store anything, it just refers to the recent catalog the coordinator sent out. And the Lender is unlikely to have its own database, it will most likely just use whatever the library is using to maintain its book collection.

You say “complexity reduction is a best practice in all forms and all levels of software development.” I agree. But when we are reducing complexity in software development, we are generally talking about simplifying our code. This is quite a different issue than reducing our overall project complexity. I assume that we are trying to do the best possible implementation job. But this does nothing to address the complexity I am looking at: the complexity of how the project is organized.

You imply that I am against code reuse. Guilty. Well, almost guilty. I believe that any attempt to reuse code must be balanced against resulting complexity. Almost all attempts to reuse code result in more complexity, and thus create more problems than they solve. By the way, the example you give of reuse is more of what I think of moving functionality into the infrastructure, which is another issue and which I endorse.

So let me summarize where I stand. I am addressing one issue only: Complexity. I am trying to understand how to best split up a project into subprojects such that
- the overall complexity of the subprojects is as low as possible
- the suprojects can be tackled as autonomously as possible
- there are as few dependencies as possible between the subprojects.

That is all I am trying to do. But in doing this, you can reduce the cost of a $10M project by 50% and increase the chances of that project being successful by 100%. Which makes this effort worth undertaking, the methodology important, and the approach unique.

That’s my two cents!

- Roger Sessions

When reviewing resumes and considering candidates, it is interesting to note that while similar experiences result in similar career progressions, not all paths end in the same destination/produce the same caliber of software professional.

Max, given your thought processes, you seem to be a great resource to have on hand. I would welcome the chance to interact more directly via e-mail or phone. Feel free to visit my Linked In profile (Michele Arndt), or contact me at michele@primestaff.com.