The Pile

How installer bundle companies solicit developers

2013-05-02T07:46:00.002-07:00

If you ever wondered how freeware/shareware developers are solicited by installer bundle companies, here's an example I received recently:

Dear Software Developer, Did you know that you could be earning money every time someone downloads your software?

In fact, hundreds of developers just like you are already turning downloads into profits through partnership with SweetPacks. My name is Limor Garten, and I am a partnership manager at SweetPacks (www.sweetpacks.com).

We create monetization opportunities from software installations through websites such asCNET (Download.com), Softonic.com and MetaInstaller. To find out about how you could earn money with every software download, please contact me directly at partnerships@sweetpacks.com or leave your details at http://lp.sweetim.com/Partners. Thanks very much,J******* M*****SweetPacks Team

It's long past time that companies who sign up for such abusive installer bundles start being penalized for harming the integrity, performance, and overall experience of countless PCs, and contributing to a general lack of trust for all third-party applications.

What are installer bundles?

For those that don't know, installer bundles are those deceptive, unwanted, additional pieces of software, often browser toolbars/add-ons, that present themselves in an intentionally easy to miss little checkbox during installation of some Windows software, particularly freeware and shareware. This terrible industry is so profitable that free, open source software is often repackaged with these bundles, then advertised and distributed by some random 'download site'.

Fundamental Flaws of Bitcoin

2013-04-27T18:59:00.004-07:00

Here I list the fundamental design flaws in the Bitcoin protocol, as I've identified them. Some are technical. Some are economic.

Let me say that there is an *army* of Bitcoin evangelicals who will argue against these points using false analogies and rebuttals far abstracted from reality. However, most serious people, even those high up in the Bitcoin food chain, do agree with these points. Unlike the evangelicals, Bitcoin developers are working to solve these issues and others, though no solutions are obvious, or are likely since these issues are so inherent to the design of Bitcoin.

In other words, many of these issues can be mitigated to some degree, but to do so compromises the principles that make Bitcoin attractive to its believers.

Regulatory Action

As we've seen, regulatory action can come swiftly and without any warning. Already Bitcoin is being abused for numerous nefarious or illegal activities. The moment we find that Bitcoins contributed to a terrorist plot, for instance, down it will go - and it will go hard! In addition to facilitating illegal activities, if Bitcoin did actually grow into what its proponents think it could be, the threat it would represent would be reason enough for governments to shut it down. Take a look at what happened in Canada, quite suddenly, on an otherwise dull Friday. And if you think governments can't take down Bitcoin, think again! They maybe can't stop all Bitcoin transactions so easily, but killing the exchanges is enough to destroy the value, and nobody will want to hold Bitcoin when they can't be converted into a 'real' currency that can be used to pay debts, such as taxes.

Fat Blockchain

The huge blockchain, a chained ledger of all Bitcoin transactions ever made, will grow forever. Already it is 6.5Gb in size, and growing rapidly. While Merkel Tree pruning can save disk space, that doesn't help with the massive amount of data that must be transferred. Could you imagine what would happen to the blockchain size if Bitcoin was adopted as the de-facto global currency? My goodness! Light wallets are a partially viable solution, though again compromising the fundamentals of Bitcoin to overcome its limitations.

Lost Bitcoins

Lost Bitcoins will erode the supply. Bitcoins are extremely limited in supply, and will eventually top out at 21 million Bitcoins some time in 2030. After that, there will be no more. Prior to that, the number mined will decrease geometrically to zero. As people lose Bitcoins, e.g. perhaps a person is in a sudden accident and dies, the available supply of Bitcoins will decrease, and the value of all other Bitcoins will increase, assuming demand remains only constant. Yes, they can be sub-divided, but that isn't the point. The point is the instability of their value will limit usefulness as a currency.

Deflation

If Bitcoins ever are a 'real' currency, they will inherently deflate since economies grow. As currency deflates, people horde it, making the situation worse. If they AREN'T used as a currency, then deflation will depend on demand outstripping supply. Economies need currency that inflates at least as fast at the economy, else the economy stagnates! Paul Krugman explained it best. People will say, "but Bitcoins can be divided into 1/100,000,000 units called Satoshis!". That will help to ensure there are always Bitcoins to circulate, but it doesn't solve the hording problem (who wants to spend something that will be worth more tomorrow?), nor does it solve the volatility problem that comes with [high] deflation. In the end, the cumulative effects of deflation probably depends on the rate of deflation.

Fraud

Scams of all types keep tarnishing the Bitcoin brand. As we all know, Bitcoins currently have value only because people are enthusiastic about it and/or believe it to be a good investment. To this day, the scams continue, and at some point the average Joe that the Bitcoin marketing machine is trying to hook will simply be left with a bad taste in their mouth!

Irreversible Transactions

There are no consumer protections what-so-ever, an intentional 'feature' of Bitcoin. Transactions are irreversible. That means that if somebody hacks your wallet and steals your Bitcoin, they are gone. If a merchant rips you off, you're out of luck (unless you want to pursue a civil lawsuit). I should note that it is possible that, in the future, stolen Bitcoins could be recovered at certain points, since every Bitcoin ever transacted is recorded in the blockchain. That's not presently the case though, and mixing services that help with anonymity also make it easy to launder money. While Bitcoin evangelicals will say irreversible transactions are a good thing, it remains to be seen what the public at large thinks of this feature. As an aside, irreversible transactions can also make refunds a bit more tedious, though this isn't a serious issue.

No Central Bank

There is, by design, no central authority that can control the supply of Bitcoin (which can't be controlled anyway with BTCs), and thus nobody to respond to price volatility or economic disruptions. You *want* someone 'at the helm', making sure that a currency is stable and available (liquid). Some will say this is an asset, but they might also change their minds when volatility or economic troubles leave them homeless!

Unregulated Market

Bitcoin is completely unregulated, and they don't want any stinkin' regulation! However, market regulations are there for a reason. Just like laws against rape and murder, you want to make sure that immoral and fraudulent activities are prevented, deterred, and/or penalized. Every unregulated market in history has been manipulated by greedy, immoral people, and Bitcoin is no different. I am speaking of the exchanges here... for small and medium sized markets like this, manipulation by savvy traders is all to easy, and they already know the tricks to use - those that have been banned from the 'traditional' exchanges!

CONCLUSION

Heed my warning here. Bitcoin is an interesting concept. It's been a fun intellectual exploration into crypto-currency and economics. You will hear a variety of defenses to all these points from the pro-Bitcoin evangelists. Keep in mind that most of them are heavily invested in the success of Bitcoin, so of course they are going to vigorously defend it. While this may just be another intellectual subject to ponder, for others it may literally make or break them! I have *extreme* concerns about people 'investing' in Bitcoin after being pumped up by the evangelicals, who will so eagerly tell them that Bitcoin are the future of all money, and will soon be worth $10000 a piece. Please, use extreme caution!

People think this is fun and games, and in their greed, continue to spread deluded arguments and hysteria, but when Bitcoin comes crashing down, the losses will be REAL and PAINFUL to many!

Comments?

- End -

---------------------------------------------
misc that maybe should be on the list

(0) Transaction Fees. Transactions already mandate a transaction fee. (I thought no transaction fees were a commonly touted feature of bitcoins?). Well, unless you want to wait God knows how long for the transaction to clear. In the future, this will be an even bigger problem, as transaction fees will be the sole motivator for work to get done, as fully described in this paper.

(1) Intermediaries Required (transaction time). that intermediaries are required for point of sale transactions to guarantee that the verification time is not slow, injecting the same intermediaries that exist in the current monetary system. Again, something Bitcoins supposedly were created to get around. Reducing the number of verifications required can accelerate the transaction time, though by increasing the risk of fraud.

(2) Malware and Botnets. Bitcoin, by chance(?), made the perfect outlet for botnets. Botnet owners may now have less cumulative mining power, but they once did, do with Litecoins, and are still trying to influence the Bitcon market through DDoS attacks. Bitcoin have incentivized botnets like never before. In order to build botnets, malware is spread. Thus, Bitcoin is encouraging malware. Further, malware is also created to target bitcoin wallets and account information. I believe, however, that this risk is mostly inherent to the development of any new financial instrument.

(3) Volatility. Bitcoin is highly volatile. This limits its usefulness. In the last week it has been relatively stable though, compared to its history, though not compared to a steady currency like the USD. However, as it stabilizes, the chance of a big return is lessened, but the risk is still there. Stability will discourage new high-risk speculators and instability will make it difficult to rely on as a 'currency' (not that it ever has been really used as a currency).

(4) Public Ledger of All Transactions. While Bitcoin may be touted as anonymous, it's not - every transaction is completely public. The sender and recipient are identified by their addresses (public keys). The blockchain is a public ledger, transmitted via the p2p network, that records (authorizes/verifies) every single transaction for every Satoshi (1/100,000,000BTC) spent for all eternity. If you get a paycheck in Bitcoins, every Satoshi can be tracked as it's spent. The identify of those address holders may not be publicly listed (or maybe they are), but the more transactions a key performs, the more ownership can be deduced, and even reasonably proven. That is to say, the people who send you money, or to whom you send money, will serve to identify your ownership of addresses. It's easy to get scared by this. Profiling based on consumption habits is even possible! There are ways to mitigate this, primarily mixing services, but it is a serious concern to some who value privacy. That said, perhaps transparency would ensure compliance and fairness when it comes to things like taxes, and prevention of money laundering.

Since every Satoshi ever spent can be tracked, stolen Bitcoins can theoretically be recovered. After all, they can NOT go ANYWHERE without being tracked! This article has an excellent perspective. The Strongcoin incident where the wallet provider recovered some Bitcoins stolen from a mining guild is a good case example. Check out this academic analysis of the proliferation of a Bitcoin theft to get an idea of how traceable Bitcoins are.

Given this trackability of every Bitcoin for all eternity, Bitcoin suddenly looks awful attractive for governance!

Fortunately, ZeroCoin is a proposed extension that overcomes this deficit, making transactions truly anonymous. However, it has some negative caveats such as much larger transaction sizes, leading to a substantially larger blockchain. One compelling feature of the proposal is that it does allow for backdoors to break the veil of anonymous transactions, so that government agencies can enforce tax compliance or combat money laundering (good things on the whole).

An Opinion on Butterfly Labs

2013-04-26T12:00:00.001-07:00

(originally typed up as a comment on a Wired article)

I truly hate to sound cynical and pessimistic, but I've been watching Butterfly Labs for a while. Hear me out on this. I think I'm right on the money. I'll try to be gentle.

The TL;DR version of this post is:

In a worst-case scenario, Butterfly Labs goes belly up and can't refund their pre-orders. In a best-case scenario, Butterfly Labs will eventually ship all its massive backlog of pre-orders, but by that time these mining rigs will be useless. In fact, if they shipped them TODAY, the flood of new ASICs on the market would quickly plummet their return. Either way, those placing pre-orders will lose their money.

People need to understand the business model BFL is using, and the fact that it is very high risk. Perhaps the worst thing a person could do today is pre-order any of their units! I don't know who would want to pay a premium to enter the bottom of a long waiting list for such a time-critical device, but I'm sure people are, else BFL wouldn't be still advertising so aggressively!

Butterfly Labs first piqued my interest because it seemed spectacular; Big claims, massive pre-orders, and a perpetually pushed back release date. Well, congratulations to them, as It seems that they have managed to use their revenue to at least create a *handful* of units.

If their supply is this low, it must mean that these are more prototypes than production units, and mass production hasn't been achieved. This means that these early units surely cost MUCH more than what they were sold for. It's worth it to Butterfly Labs though, as it silences the skeptics and gets them even more pre-order revenue.

As they continue to take thousands of pre-orders worth millions of dolars, I can't help but wonder how they plan to ever deliver that number of devices.

Pre-ordering a unit that's going to market is one thing, but what people are doing here is financing the entire enterprise. That's risky for everyone! Now, Butterfly Labs *may be* in a position to make enough revenue from mining to supply refunds to all those who give up on waiting, or to whom they can never possibly deliver! Otherwise, they *may be* in trouble! The fact that they seem eager to generate more pre-orders is not a good sign!

Do you realize how easy it would be to 'get behind' when you've taken on so many pre-orders for a device you haven't built yet? Who knows if they can manufacture them as cheap as they hoped (probably can't), and thus they will be taking a loss on, at least, all the early orders. If they do NOT make this deficit up by mining or whatever, then those who ordered last *will* lose their money.

Perhaps they know they are in trouble. After all, if Butterfly Labs was really more concerned about order fulfillment than generating more pre-order revenue, it seems like they'd have sent these units to customers that has been waiting for nearly a year, rather than to Wired Magazine and other publicists. By sending units to publicists, and continuing to spend big bucks on advertising, they are creating an even BIGGER BACKLOG, something a company on the up and up wouldn't seemingly want to do.

It's very much like a ponzi scheme. Remember, those ordering these devices are doing so because they believe it will offer a good return on their money! It *IS* an investment! Thinking about it in that context makes things clear. You think you'll be rewarded for this investment, and a very few who got in early may be. Everyone else is financing the development of these prototype units that get sent to a few lucky recipients. Most will never see their order fulfilled, at least certainly not in a time frame that makes it a worthwhile investment!

If I had an order pending, I'd ask for a refund while they are able to provide them!

You tell me - how is this NOT true?

Binary Interleaving: A technique to obfuscate data structures

2013-04-14T23:29:00.004-07:00

Binary Interleaving (or Data Interleaving) is a term I made up to reflect an idea I've been contemplating lately; interleaving the bits of a set of variables into a single binary blob. Any number and types of variables could be interleaved together. Access to variables in the interleaved blob can even be on-demand, with a controller class encoding or decoding variables on the fly.

What in the World?

Data Interleaving is the process of translating any number of variables to a single binary blob by interleaving the bits of the variables. This obfuscates the variables in memory or external storage. The entire blob need not be decoded to access member variables, though it can be for improved performance.

Why?

This will help complicate reverse engineering of code. It will particularly deter identifying data types and variables. Plaintext is also well obfuscated with this interleave.

Interleave Map

The variables to be encoded could be defined by an array of byte sizes of those variables and, optionally, pointers to a location in memory to retrieve or store their reconstituted form. In the case of on-demand access to an interleaved blob, individual variables can be decoded and re-encoded on the fly, so buffers for reconstituted storage are optional (though they may be temporarily reconstituted by the controller class as members are modified).

The members of the bitwise interleave can be referenced in the source code via their indices. For instance, index 0 may be MY_VARIABLE_INSTANCE. By passing the variable index to an interleave blob controller class, it knows the size and, optionally, a pointer for constituted storage.

Member data types can be anything. They need not be similar. When one variable ends, it is simply ended. See a few paragraphs below for what happens when a single variable is longer than the others.

/* member information */
/* optional pointer to its normal, constituted storage location */
/* (for use in encoding and decoding the member) */
/* and the size of the member */
class CInterleaveMember
{
void *pvConstitutedStore;
unsigned long nMemberByteSize;
};

/* INTERLEAVE MAP */
CInterleaveMember aInterleaveMap[]
{ szSomeString, sizeof(szSomeString) },
{ &nIntegerMan, sizeof(nIntegerMan) },
{ &cMyClass , sizeof(cMyClass) };

void *pBLOB; /* interleaved data stored in a allocated blob */

The total size of the blob need not be stored, as it is the sum of all member sizes in the interleave map. The interleave map provides everything we need to know.

The Process

In case it is not clear, the process for the interleave would go something like this: The array of members is 'walked', putting or getting the current bit index from each member variable, advancing to the next bit index after the entire array has been walked. When a member variable is full of bits (exhausted), it is skipped in subsequent interleave iterations (more on long vars later).

For simplicity, let me define a few variables in bits only (not matching above):

szSomeString 0 1 1 1 0 0 1 0
nIntegerMan 1 1 1 0 0 0 1 1 1 0 0 1 0 0 0 1
cMyClass 0 0 0 1

For the interleave, a bit is taken from each variable in succession.

First iteration of the interleave, get first bit from each ...
0 1 0
Next iteration(s), get the next bit from each ...
0 1 0 1 1 0
0 1 0 1 1 0 1 1 0

...

When a Member is Longer than the Others

In the case where one variable is much longer than the others, thus having no pair to encode with, one could use a simple XOR, and/or toss in redundant, unused data from the prior members. Any number of strategies are possible to prevent plaintext storage in the case of an abnormally long variable not having an interleave partner for its ending bits.

Sample Code

For example, the following represents a high-level view of the calls to a fictional class facilitating Binary Interleaving:

/* PROTECTED VARIABLES */
/* These get stored in an bitwise interleave in the binary blob */
char szSomeString = "Is there anybody out there?";
unsigned long nIntegerMan = 0x9090;
MyClass cMyClass("whoopie");

class CInterleaveMember
{
void *pvConstitutedStore;
unsigned long nMemberByteSize;
};

/* INTERLEAVE MAP */
CInterleaveMember aInterleaveMap[]
{ szSomeString, sizeof(szSomeString) },
{ &nIntegerMan, sizeof(nIntegerMan) },
{ &cMyClass , sizeof(cMyClass) };

/* NOTE: Total blob size is the of members of Interleave Map */

/* INTERLEAVE REFS */
typedef enum
{
_szSomeString=0,
_nIntegerMan,
_cMyClass,
} InterleavedVariables;

void *pBinaryBlob; /* dynamically allocated blob storage */

/* Fictional class constructor, passing the interleave map to it */
/* From the interleave map, it can calculate the total blob size */
/* then dynamically allocate storage for the blob. */
CBitInterleaver cBitInterleave(aInterleaveMap);

/* If the blob is externally loaded, or needs ext stored, we */
/* may need to get access to the blob buffer. Fictional example: */
/* We know the blob size from map! The input size is for safety. */
cBitInterleave.SetBlob(pIncomingBlob, nSrcBufferSize);

/* Or we can get the blob */
nBlobSize=cBitInterleave.GetBlob(ppOutgoingBlob);

/* Example to encode or decode the entire blob to constituted */
/* storage. We already provided the map, and it decodes or encode*/
/* to the listed pointers.
cBitInterleave.EncodeBlob();
cBitInterleave.DecodeBlob();

/* Example call to decode a member of the array */
/* We pass it the INDEX into the MAP, and dest buffer */
/* From the Index of _nIntegerman, we ALREADY know the size */
/* The out size is for safety. */
cBitInterleave.GetVariable(_nIntegerMan, &nIntegerMan, sizeof(nIntegerMan));

/* OR we can use the default storage address in interleave map */
cBitInterleave.GetVariable(_nIntegerMan);
/* Example call to encode a member of the array */
/* We pass it the INDEX into the MAP, and input reference */
cBitInterleave.SetVariable(_szSomeString, &szSomeString, sizeof(szSomeString));

Feedly and RSS Subscription Extension

2013-04-04T15:25:00.006-07:00

If you use the RSS Subscription Extension for Chrome and want Feedly support, here's how. As some may know, this extension adds a feed icon to any page that offers up RSS feeds. This allows you to easily click to subscribe.

When you click on the icon, there is a drop-down list of available news readers. Right now, only a couple are listed, after the announced death of Google Reader. You can, however, add a new one. To do this, you'll simply need the URL that is used to add feeds to the reader. Chrome inserts the RSS feed URL via a variable replacement (%s). If you don't see this list, you may have selected a default reader, and can access the list via the Options for this extension.

The URL you need is:

http://www.feedly.com/home#subscription/feed/%s[action.subscribe

Just paste it in when you click 'Add', and name the reader whatever (e.g. feedly). From that point forward, you have easy access to RSS feeds.

Apache mod_qos Server Status

2013-04-03T17:12:00.001-07:00

For those that haven't seen mod_qos's server_status page, it's pretty nifty, and allows for some operations right there on the Apache server-status page! This page is, of course, accessible over HTTP at /server-status. for Apache servers where it has not been disabled, and you have access. For instance, http://127.0.0.1/server-status

It is a shame that mod_qos has not been ported to Apache 2.4 yet. I am not sure what blockers exist, or if it has been deprecated in its entirety. Perhaps a reader will know...

Google Analytics April Fool's - Visits from the International Space Station

2013-03-31T17:55:00.000-07:00

The Google Analytics team has pumped out an April Fool's joke already. They are showing prominently in the center of Google Analytics Real-Time View the International Space Station Control Room. See attached picture. When you check the page the visitor(s) are supposedly hitting, it is April Fool's.

The Abysmal State of Security Software

2013-03-23T12:09:00.003-07:00

Security software, specifically anti-virus and anti-malware software, has been a staple in the Windows world since Windows 95. Users are told they need this software to keep them safe, or else they're in for a world of hurt. In this post, I'll take a minute to tell the real story and reveal why your security may make you considerably LESS safe!

Security software rarely detects new or targeted threats

Sadly, the virus and malware authors are always one step ahead of security industry. These rogue programmers use the same software you do, and actively work to make sure they have defeated it. Since differentiating new malware from legitimate applications is nearly impossible, new malware usually slides right through the detection net. After all, if security software worked great, a lot fewer cases of malware infestation would exist!

But wait, you say, what about those 99% detection rate claims? Well, they are testing against samples already known to the security industry. I would certainly hope they have a good detection rate when it comes to those! For unknown , new, or targeted malware, which is regenerated daily, and the only kinds you are likely to encounter, the detection rate is much lower.

Security software is prone to false positives

In the effort to try to detect new, unknown, or targeted malware, some security products are well known to alert on just about everything, including lots of legitimate applications, especially those from smaller developers. As a small developer myself, I find this highly frustrating. I sign my applications, make sure they don't do anything that looks shady, and generally do all I can to avoid false positives, but still they occur so routinely they are to be expected.

It isn't just small developers that are affected though, false positives occur on all sorts of software. One false positive a few years ago, on svchost.exe, a critical part of Windows, had a catastrophic effect on countless PCs worldwide.

Worst is that the new web site rating services that security products now offer can take a single false positive and turn it into a badly rated domain. Getting these false positives or web site ratings fixed can be very difficult and time consuming. Some security companies are responsive, others much less so! I had one invalid site rating from a major security provider, which happened due to a false positive, last weeks, and then recur 4 times before they finally fixed it. I nearly lost my mind.

Better to have false positives than missed detections, you say? Well, unfortunately, it doesn't work that way. The malware authors work to avoid detection, so are somewhat less vulnerable to false positives. Further, users get so used to seeing false positives, they may very well quit taking detections seriously!

Security software companies often distribute malware themselves
(installer bundles)

Since I consider installer bundles malware, it is painful to see security companies using installer bundles. We've all seen these bundles. You download application X, and are presented with deceptively packaged offers for applications Y and Z. The user's intent was only to install application X, so it is a clear violation of the user's wishes.

It is extremely easy to accidentally get one of these bundled components installed. Since all parties involved make money per install, they have gotten more and more deceptive. Download sites like CNET now even attach their own bundles to downloads.

The most common bundles are toolbars and other web browser add-ons. They clutter up your PC and web browser, bringing performance down. Some are difficult to disable, and almost all behave in deceptive ways. This massive browser add-on problem got so bad, with some users ending up with an unreal number of browser add-ons, Microsoft had to start disabling all of them by default in Internet Explorer, forcing users to selectively choose which are enabled.

Sadly, the entire software industry seems to have adopted these bundles. While it would be nice for security software to detect these as malware, instead security software companies are themselves using installer bindes! They distribute free scanners, web site rating tools, and other 'teaser' components with common applications. One example is McAfee bundling its teaser products with Java!

I wish I could say there was ANY security company out there detecting installer bundles as malware, but there aren't. Instead, I can't think of one that does NOT use installer bundles.

Security software companies have a hard time deciding what is malware

Similar to not detecting deceitful installer bundles as malware, security companies have a hard time deciding what is malware, and what is just a deceitful application. They've even been sued. These days, rogue software companies simply push it up to the limit of being considered malware, and get away with virtual murder. Their borderline applications not only aren't detected, but the distributing web sites are often certified by security companies as safe, a service you can pay for at most security software companies.

Given that rogue software of all types gets away intentionally undetected, the utility of anti-malware products goes down even further.

Security software offers a false sense of security

Given all this, we see that security software isn't very useful. However, it really becomes harmful when users believe they are protected from threats. This may leave them to act more wrecklessly, under the false notion that their security software is protecting them. It seems preferable to remove this illusion of security, and instead have users realize that their safety is in their own hands. User education and common sense is much more effective than any security software!

Conclusion

If slowing your PC down considerably wasn't bad enough, we now see that the actual utility of security software is quite questionable, as are the practices of many of these companies when it comes to rogue installer bundles. They are unlikely to detect any real threats, likely to let rogue borderline applications skate by, and give users a false sense of security. I'd say, toss away the illusion, and start realizing that nothing can protect you except your own judiciousness!

-- End --

appendum #1
Although you won't see it until something is detected, did you know that Microsoft Security Essentials is now part of Windows Defender, and built into Windows 8? Yes, that's right, you need NOT install any third-party security software! Windows Defender is arguably the best option people have right now, as it has a low false positive rate, is efficient, and not obtrusive. In fact, most people have no idea it is there! Further, Microsoft doesn't use installer bundles, as far as I know.

appendum #2
I believe in the long-term, we'll move to entirely store/approved applications, which will allow tracking and rating by vendor history. The problem is, the definition of malware does not include 'boderline' applications that are just plain deceitful, thus we'll always have crapware! Until corporations start acting with more moral fiber, things will continue to be much the same.

The Software Review Business or Pay or Get Bad Reviews?

2013-01-07T00:29:00.001-08:00

The Software Review Business
or Pay or Get Bad Reviews?

I received an email (then two more telling me to 'think about it' and 'think more about it') ... What follows is what I believe a pretty clear extortion attempt, only thinly veiled.

Currently i no longer post to GOTD. (Persona non grata) and normally i would not
directly contact a vendor (tho, i'm considering changing this - i may start selling my
opinions/advice...) but in your case i'm going to make the effort. ((Half for "free" and
the other 'half' will cost you a thousand dollars - but don't fret i don't even have my
PayPal set up yet...))

free half:

After several disappointing things i have removed your product. After several
unacceptable things i have posted this email. The next time you present on GOTD i
will make every effort to dissuade others from installing your product.
(ps btw: these are valid complaints that would warrant attention from a
concerned vendor - need i say less?)

paid half:

(will be sent upon receipt of payment.)

This is my personal interpretation of this email. I do not claim that I am correct in this interpretation. Judge for yourself. Fortunately I'm too broke to be extorted, so the decision is pretty easy. I thank all those who help defend my software against attacks like this. Now we know that software reviews have turned into a business.

If someone is extorting negative reviews, they are surely selling positive reviews! That makes this the latest online business of manipulation.

I long ago proposed that download sites use more means to verify posters, ala Angie's List. Let's hope they do. I'm all for real reviews, even if they aren't all flattering.

Why Windows is akin to the wild, wild west

2012-11-23T01:37:00.008-08:00

One of the problems as a Windows developer is that Windows violates many core principles of a protected mode Operating System. Namely, that applications should be isolated from one another. The ease of DLL injection via shell hooks is simply absurd, as is users running with administrative rights. Although UAC helped a little with the latter, it didn't do much since any software installation requires elevated rights. During installation, whatever happens, happens ...

Now, this wouldn't be a problem if there wasn't so much crap out there. There are two issues. One is poorly written applications that map themselves into other processes, causing problems throughout the system, sometimes quite randomly, or sometimes very selectively.

The other issue is malware or other unwanted applications like pay-per-install toolbars. These things get bundled up with software that is often free and open source. Download sites then pay for advertisements and fight for search engine rankings. It is legal for them to do this with free and open source software, regardless of what the author wishes (depending on the license). For commercial software, the vendor can at least protest. Of course, since a certain otherwise well respected download site set the precedent for allowing bundles, or download managers, this has become more common than ever, and something that can only be expected from most download sites that host their own files. The few exceptions out there are gems and we'll have to hope they don't give in to the financial temptation.

So, we end up with an environment which is absolute chaos. Any PC that gets anything installed, or used for that matter, is going to likely end up, over time, in a state of severe degradation, if not being totally compromised from a security standpoint (0wned ;p). Since malware, once installed, can hide its presence completely from any scanners while the OS is booted, your security software won't do much after the fact, and usually doesn't do much before the fact. In fact, false positives are a huge problem because they are trying so hard to find some way to determine what bad software even looks like. There is no pattern to it, as there was with viruses.

This may change, to some degree, with Windows 8 and the new Microsoft App Store, assuming it takes off. Though these stores have proven to be highly exploited as well, since it is difficult for them to define what a 'bad' application is, and therefore you end up with shady vendors operating in every shade of grey they can. Microsoft does at least seem to be taking the vendor verification process seriously, as we at Bitsum had to undergo pretty strict integrity checks.

Oh well ... that's just how it is.

UPDATE:

Do remember that a more strict application ecosystem does mean a lessening of freedom. Just like some vendors are now locking the OS on the device, applications you might want to use that aren't certified won't be available!

Bug discovered in Windows 8/2012 Taskbar (traditional shell / UI)

2012-10-24T23:33:00.002-07:00

There is a bug with the Windows 8 Taskbar in that it refuses to refresh at all until a swap to Metro is done. At that time it will do a one time refresh. It stays persistently frozen and/or out of sync, including the clock, running application window icons, and the system tray. Context menus show the wrong items, indicating that Windows know what the Taskbar should look like. This suggests it is some sort of display problem.

Interestingly, when I installed StartIsBack (http://startisback.com) to restore the Windows Start Menu in the most authentic (and simple) way possible, it seemed to immediately fix this on a system that had been experiencing a frozen task bar at the time (clock frozen until Windows key hit, etc..). I will report if this is a long term work-around in time.

The return of native code Windows apps

2012-10-04T23:10:00.000-07:00

Microsoft has been strongly encouraging the use of C++, native code, and even use of C++ for managed code. It seems they are poised to push C++ ahead of C#, perhaps because they know they need more efficient code for low-power mobile devices. Anyone who would say that C++ is a language past its prime has no idea what they are talking about, that's for sure. Despite the fact that all popular modern OSes are written in C or C++, application developers continue to favor C++, as do users.

The good news? These days we have what I like to call self-managed, unmanaged code. That is, unmanaged (native) code that watches itself so closely that it performs like its own CLR manager, preventing errors and cleaning up any leftover debris (or at least notifying the developer of the condition). The basic CRT has even gotten more sophisticated, and secure. Take C++/CX for instance...

Here's a recent article on developing native C++ applications for Metro/Modern UI using XAML and native/unmanaged C++/CX style code: http://msdn.microsoft.com/library/windows/apps/hh465045.aspx .. C++/CX being the latest thing (essentially C++/CLI syntax that compiles to native code). Clearly they are thinking they need native apps .. I figure it is for the low-power devices and the realization that some applications will need maximal performance. Otherwise, C# would be the thing to push, as for managed code development, it is much easier.

Other examples of the switch to unmanaged code can be seen in a variety of Microsoft applications (e.g. Microsoft Security Essentials, now Windows Defender), and those of other large corporations. AMD, for instance, switched the Catalyst drivers FROM .NET to native code several years ago.

Recommending Open Hardware Monitor

2012-10-03T17:35:00.003-07:00

While looking for an alternative to certain basic system monitoring software that has become 'infested', I found Open Hardware Monitor . It is written in C#, thus utilizing the .NET CLR. It also works under Linux /w MONO installed. Even better, it is open source! Pretty cool! We've seen such efforts fail before though ...

Keep the project open to the community and *nobody* benefiting from the mere existence of the application!

The only problems are in this project's future. It is sure to be the 'new thing' to monitor your system with. Now, what have we learned from previous system monitoring utilities?

The authors are likely broke because they are author freeware, thus are targets for the commercial 'sharks'.
Little by little, the author(s) sell their soul, with advertisements, then 'bundles', then more and more bundles.
Rogue download sites repackage the freeware and get money without ever paying the developer(s).
ANY of the above leads to stagnation as the developer(s) all feel unfairly subsidized.

How can this project be protected? Well, I am proud to say that I protected one or two free, open source projects from the complications of commercialization. The key is simply not be out to make money. Don't allow donation buttons. Don't have a treasurer. Users can give donations to individual developers, if they want, in secret, perhaps for implementing X, Y, or Z. Keep the project open to the community and nobody benefiting from the mere existence of the application!

How to fix: Windows 8 Modern UI if stuck on the wrong monitor

2012-09-22T16:13:00.004-07:00

Us traditional interface users may not pay much attention to the Modern UI, but the other day I managed to get it 'stuck' on a monitor I didn't want it on. It popped up there, whenever invokved, with no way I knew of to move it. Finally I Google'd and discovered if you hover in the lower _left_ of a screen you can select to open the Modern UI on that monitor. From that point forward, it remembers its last monitor. While this is quite trivial, who knows - maybe it saves someone else some time. There has been lots to get used to with Windows 8, and while I consider it a great OS, I wish Microsoft would have spent more resources on marketing and user education (or maybe they plan to).

CPU Overheating Legacy Mode

2012-09-07T08:43:00.004-07:00

CPU Overheating in Legacy Mode

When in Legacy (non-protected) mode it seems some CPUs are particularly susceptible to overheating. This is before they start frequency scaling, sometimes before dynamic FAN speed control, perhaps before use of the HLT instruction, and definitely before use of any core parking. Thus, these mitigation strategies may be all that keeps some CPUs from overheating. I've always noticed an increase in thermal emissions when my CPU is running full speed in legacy mode, but today is the first time I accidentally spent too much time in legacy mode, causing my system to shutdown pre-boot due to the CPU temperature (which had raised to 74C, though post-boot, with mitigation strategies in place, is back at 45C). Now, this is actually an indication of some trouble on this PC, despite the heavy load I place on it. Either I must allow for such extreme temperatures, or perhaps re-seat the Heatsink. Still, on a PC with a heavy lead that has no issues running continuously at 100% CPU frequency in High Performance mode, this is a surprising - and scary - discovery. I can't help but wonder, how long would your PC last in legacy mode?

UPDATE: In my case, since I was in a pre-boot RAID configuration tool, I believe the failure of the CPU fan speed to ramp up may have been the primary cause. The UEFI was likely not fully initialized at this stage, and thus left me in a vulnerable state, while the CPU ran at 100% of its clock speed, with 100% utilization of a single core at least (as it does in legacy mode).

UPDATE2: I have reproduced this many time by entering the pre-boot configuration on a HighPoint RAID controller. During its configuration interface, either the fan isn't running fast enough, or other cooling mitigation technologies aren't working. CPU temperature quickly escalates, and will eventually halt the PC. Otherwise, in the temperature in between, it will warn you at next bootup if it hasn't cooled already.

Keeping a process below a certain total CPU % use

2012-09-06T16:33:00.000-07:00

Keeping an process below a certain total CPU % use

So, you want to keep a process limited to using only a certain percentage of all available CPU time? For single core systems, it used to be I'd tell people this wasn't recommended, but if they really had to forcibly limit a process, then they would have to use Process Lasso's unsafe 'Hard Throttling'. Fortunately, these days, if you've got a multi-core system, you are in luck! The more cores, the more control you'll have.

By limiting the CPU affinity of a process to specific cores, you have the ability to control the available CPU time it has access to, out of the total CPU time pool. For example, in an optimal world of a quad core system with 4 fully independent physical cores, you can set increments 25%, 50%, 75%, or 100%. Now, when you throw in logical processors, the picture gets more complex. Since the logical processors won't perform well at all, the % limits are actually staggered in *effect*. While the scheduler may show an exact 50% for 2 of 4 logical cores (100% of the 2 cores time is being used), that may not mean those two cores are executing at 100% the capacity of two physical cores. In other words, the CPU time is used, but performance takes a hit. This is hard to quantify for logical processors, but ThreadRacer can help.

Remember, Intel has its HyperThreaded cores staggered, starting at the second core (core #1 if core #0 counted as the first), and goes from there. Similarly, AMD Bulldozer+ Modules have adjacent processor cores that share computational units. Processors #0 and #1 would share some computational units, so you wouldn't want to load them both up if other cores are available. The scheduler itself will try to avoid such situations, in fact. With this 'every other core' method, it is easy to distinguish processors that are co-dependent (AMD), or HyperThreaded (INTEL).

For the AMD Phenom II and below, all cores are true, physically independent cores, making the Phenom II the last processor made in this fashion. That is why the AMD Phenom II x6 continues to shine in benchmarks against the latest AMD generation, despite a deficiency in base clock speed. It has 6 fully independent cores, something no current AMD processor has - they instead of 8 co-dependent cores, counted as 4 physical cores and 4 additional logical cores in marketing and by the Windows OS CPU Scheduler.

You can do this with:

1. Process Lasso's Default CPU Affinity,
2. When ProBalance events occur
3. Via the Process Lasso Watchdog (so that the change is only induced when certain criteria are met)

More information is available here: When CPU Affinity Matters.

JAVA Update Check Frequency - Once a Month by Default - How to Change it, and Why You Should

2012-09-05T20:59:00.001-07:00

JAVA Update Check Frequency - Once a Month by Default

How to Change it, and Why you Should

I just checked the default JAVA Update Check Frequency. Turns out it is EVERY MONTH. I figured this needs an adjustment, given what we now know! There should be several more 0-day updates necessary to protect yourself from immediate exploitation by simply visiting the wrong site! To change this, you need to find the "Java (32-bit)" shortcut. Be sure to 'Run as Admin', else it will - without error - not actually apply your changes!

The actual executable backing the shortcut is in %PROGRAMFILESDIR%\Java\bin\javacpl.exe (Java Control Panel). That's "Program Files" for 32-bit Windows installations, or "Program Files (x86)" for 64-bit Windows installations.

Note that this may not *always* matter, as the browser you use may have its own internal implementation of Java, and/or enforce Java updates more regularly. However, I figure, why take the chance?

Increasing PC Performance while Saving Energy through Process Lasso's Energy Saver

2012-09-05T19:17:00.002-07:00

What is the Process Lasso Energy Saver?

I love neat little innovations that were left out of Windows, and one such innovation is the ability to simply run in a High Performance power profile, while dropping automatically back down to Balanced or Power Saver when you go Idle, then back up again when you return. It makes perfect sense! That is Process Lasso's Energy Saver.

Why was an Energy Saver like mechanism left out of Windows?

The reason this is not always do-able by default is because *some processes* likely need excluded to attain the maximum performance you want in your highest power profile (named High Performance on English PCs by default).

After installing Process Lasso, simply turn on Energy Saver and you are done! Best of all, Energy Saver is one of the many Process Lasso features available for *free* indefinitely!

Enjoy!

Misc NT 6.2 screenshots (Windows 8 / Windows 2012)

2012-08-25T12:35:00.003-07:00

From Process Lasso, showing DWM running in its own user context.

The Visual Studio 2012 Migration - and XP/2003 (NT5)

2012-08-18T13:33:00.001-07:00

I am always an early adopter, I get excited about technology. As a developer, I should perhaps be more pragmatic. When Visual Studio 2012 went RC, I switched to it. I wanted to be ready, after all. There weren't many complications, but one was a complication... the initial (soon to be redacted) lack of support for Windows XP or Windows 2003 as a build target.

Now, I have my ways to get around it, namely a utility I'm about to upload called PESetVersion. This will let me set any of the versions of a PE/PE+, though the only applicable one is the SubSystem version. By changing this, I hope to achieve compatibility with XP and Windows 2003. Now, the CRT isn't guaranteed to function right, so I guess we'll find out in extended testing. If all runs, there is a 99% chance it will be fine. EDIT: It didn't (see below), the CRT uses NT6 APIs.

I would imagine the CRT doesn't do anything not compatible with NT 5, unless it has some explicit check. My reasoning is because the developers of Visual Studio 2012 actually planned to keep XP support.

From what I hear, the marketing or some management arm nixed it from the final product. BUT, due to developer demand, this decision was redacted. Support for Windows XP and Windows 2003 will not be in the RTM though, it will come in some out-of-band update 'later this Fall'

Prior to VS2012 there was VS2010, which similarly dropped support for Windows 2000. However, you could still use an older Platform Toolset (compiler, linker, etc..) as specified in the project configuration.

Sadly, the subsystem version you specify in the linker appears to be neutered so that you can't over-ride its designation of NT 6.0. It even says '5.0' in the configuration by default, but this is entirely ignored.

This will be part of the PESuite Tools authored by Bitsum. It should be released in the next 24 hours.

UPDATE: It appears at least in some modules the CRT does import some NT6+ APIs. You therefore must use an alternate CRT or patch the existing CRT. The ability to use an older VC9 or VC10 Platform Toolset is also there, and the preferred solution at this time ;o.

Cleaning up registry debris the safe way ...

2012-08-03T05:13:00.000-07:00

As you know, many programs leave debris around after uninstall. Now, there are all types of debris, and some of it is more important than others. As I recommend on my RegMerge page, do NOT resort to registry cleaners that may very well cause more harm. Instead, I recommend using RegEdit to search for the product's name, then delete keys or values that have it. WARNING: Be sure you know what you are doing and have made a backup of your hives or restore point first.

The most serious thing left behind are shell extensions or unrestored file associations. These can cause complications, or at least annoyances. In the latter case, unrestored file associations (e.g. the program didn't revert to associate whatever extension with the previous program), you must manually correct these. I've always thought of a project to try to repair these to their default state, and indeed RegMerge was/is the basis for that. However, for it to work, we first need 'default sets', which are .REG repair files that simply replace the shell association with its default handler. If you have such, please submit them, you'll of course get full credit for your contribution! Remember, RegMerge is 100% freeware from Bitsum - one of our many contributions to the community!

Decrypting EFS

2012-07-20T12:49:00.002-07:00

One of the many pains of NTFS's EFS is that if you decide to no longer use it, you can get in a situation where you are waiting hours and hours for your HDD or SSD to decrypt each file and folder, and change its attributes.

Worst, if you abort that operation, or have inaccessible files, you can end up with MIXED filesytems, with some files encrypted, and others not. This can cause complications with some backup solutions that use direct copies (if the destination can't be encrypted, for example).

So, how to fix? Well, you could toggle that Encrypted attribute checkbox in the folder properties, start to encrypt, then abort. Then do the opposite, being sure to include all subdirectories and files.

OR you can use the CIPHER tool, built into Windows. Cipher let's you get and control all sorts of EFS information. Anything EFS related - it has. Including what we want, decrypting an entire directory tree.

For example, say I want to make sure there are no EFS files left on drive D:\. The command might be:

CIPHER.EXE /D /Sd:\

In my case, on a drive I had 21 'stray' EFS files and folders still encrypted. Fixed now.

Run Cipher /? to get a full list of options.

Why is EFS bad? One, the filename problem. Filenames are visible. Also, since they let the attacker know likely points of plaintext (e.g. this plaintext data is likely at location X in file type Y), the key becomes quite attackable. Then there is the issue of backing up your certificate, which has been made more clear in Vista and above, but can still be a bit of a pain. TrueCrypt or BitLocker are definitely the thing to use. I prefer TrueCrypt myself, one of the best F/OSS applications ever created. EFS, in my mind, is an illusion of security more likely to cause data loss than protect your privacy.

Deter SQL injections with mod_qos?

2012-07-20T02:37:00.003-07:00

I updated my previous post about Apache's mos_qos to include one additional example use.

This might help to avoid an SQL injection attack, who knows. Now, it is definitely NOT absolute security. It likely won't help you from anything other than some automated probes, and maybe it happens to save you from a particular SQL injection vulnerability. The idea is simply not to allow certain SQL commands in the HTTP request.

Now, this means it should be used selectively. If you use it on a whole site, something may break. Therefore, encapsulate it in Directory/Location tags.

# don't allow a certain SQL command string patterns within the HTTP request
# !WARNING: use location or directory tags, probably not good for whole server
QS_DenyQueryBody              on
QS_DenyQuery       +s01       deny "(EXEC|SELECT|INSERT|UPDATE|DELETE)"

Classic Tools: DPC Latency Checker

2012-07-20T02:28:00.001-07:00

Classic Tools

Every once and a while I'm going to post about a utility that has been around a while, but we still love and use it. Some of these, due to lack of maintenance, have been 'forgotten' in time. Let's bring a few back to life.

Today's subject: DPC Latency Checker v1.3.0 . This is 100% freeware. There is definitely no profit incentive here. This developer clearly just wanted to contribute to the world, and did so. Then he moved on.

What he left is really a 'perfect' little utility for what it does. It continues to 'just work' without maintenance in years (afaik).

Download: http://www.thesycon.de/deu/latency_check.shtml

Apache's mod_qos - Your savior

2012-07-14T03:18:00.002-07:00

I've posted about Apache's mod_qos before, but I wanted to post about it again. The sheer number of features this module offers are astounding. You can limit connections per IP, give higher priority to certain clients or applications, and generally control the workload of the server. It is what Apache always needed - indeed, what any web browser needed. I won't bore you with the details, but after you've either statically linked to this module in your build, or loaded it up dynamically, you can use configurations like below, as seen on the mod_qos documentation page.

Feature set:

The maximum number of concurrent requests to a location/resource (URL) or virtual host.
Limitation of the bandwidth such as the maximum allowed number of requests per second to an URL or the maximum/minimum of downloaded kbytes per second.
Limits the number of request events per second (special request conditions).
It can also "detect" very important persons (VIP) which may access the web server without or with fewer restrictions.
Generic request line and header filter to deny unauthorized operations.
Request body data limitation and filtering (requires mod_parp).
Limitations on the TCP connection level, e.g., the maximum number of allowed connections from a single IP source address or dynamic keep-alive control.
Prefers known IP addresses when server runs out of free TCP connections.

EXAMPLE:

# maximum number of active TCP connections is limited to 896 (limited
# by the available memory, adjust the settings according to the used
# hardware):
MaxClients               256
# idle timeout:
Timeout                  60

# keep alive disabled after we start to fill up
KeepAlive                on
MaxKeepAliveRequests     60
KeepAliveTimeout         3
QS_SrvMaxConnClose       196

# don't allow more than 30 TCP connections per client source address:
QS_SrvMaxConnPerIP       30
# name of the HTTP response header which marks preferred clients (this
# may be used to let the application decide which clients are "good" and
# have higher privileges, e.g. authenticated users. you may also use
# the QS_VipUser directive when using an Apache authentication module such
# as mod_auth_basic or mod_auth_oid):
QS_VipIPHeaderName       mod-qos-login

# enables the known client prefer mode (server allows new TCP connections
# from known/good clients only when is has more than 716 open TCP connections):
QS_ClientPrefer          80

# minimum request/response speed (deny slow clients blocking the server, 
# e.g. defending slowloris):
QS_SrvMinDataRate        120 1500 400

# and limit request line, header and body:
LimitRequestLine         7168
LimitRequestFields       30
QS_LimitRequestBody      102400

# block clients violating some basic rules frequently (don't allows more than 20
# violations within 5 minutes):
QS_ClientEventBlockCount 20 300
QS_SetEnvIfStatus        400               QS_Block
QS_SetEnvIfStatus        401               QS_Block
QS_SetEnvIfStatus        403               QS_Block
QS_SetEnvIfStatus        404               QS_Block
QS_SetEnvIfStatus        405               QS_Block
QS_SetEnvIfStatus        406               QS_Block
QS_SetEnvIfStatus        408               QS_Block
QS_SetEnvIfStatus        411               QS_Block
QS_SetEnvIfStatus        413               QS_Block
QS_SetEnvIfStatus        414               QS_Block
QS_SetEnvIfStatus        417               QS_Block
QS_SetEnvIfStatus        500               QS_Block
QS_SetEnvIfStatus        503               QS_Block
QS_SetEnvIfStatus        505               QS_Block
QS_SetEnvIfStatus        QS_SrvMinDataRate QS_Block
QS_SetEnvIfStatus        NullConnection    QS_Block
# even a little anti-SQL injection ... 

# don't allow a certain string pattern within the request query or
# use location or directory tags, probably not good for whole server
QS_DenyQueryBody              on
QS_DenyQuery       +s01       deny "(EXEC|SELECT|INSERT|UPDATE|DELETE)"