tag:blogger.com,1999:blog-60724711697489580292024-03-27T03:38:22.818-03:00devblogLet's make computers do amazing things!Ronoaldo Pereirahttp://www.blogger.com/profile/02620222149209242748noreply@blogger.comBlogger10125tag:blogger.com,1999:blog-6072471169748958029.post-59695775621435100022022-05-16T13:19:00.004-03:002022-07-15T17:19:34.510-03:00Export your assets with this simple gcloud command<p>Manage Google Cloud environments is often a complex task. You need to take care of the resource provisioning and keep all services up and running, but also may need understand the service usage and by proxy their costs, who is using your resources and what projects have certain APIs active. This escalates quickly as your company adopts more and more Cloud services, and you end up with a very high risk to have some shadow IT in the cloud.</p><p>One of the tasks you may need to do is to <b>visualize all your Cloud Assets</b> and, eventually, answer some specific questions on them, like "which project has a specific API enabled?". An <i>asset</i> in this context is <i>any resource</i> you may have created using any of the APIs, in any of your organization projects.</p><p>Cloud Asset Inventory is available from the <b><a href="https://console.cloud.google.com/iam-admin/asset-inventory/dashboard" rel="nofollow" target="_blank">IAM > Asset Inventory</a></b> page on Cloud Console. From there, you can see all your cloud resources at a glance using a pretty geographic visualization:</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhoAABmNJ04IBfwcuZF9dVH6ujw1BIckfTJydHLHJdSlZueCXIMc-ad7VzQ-APyWP93ISOE6XplO8QsfaA0948Q5PD7HMBclvQhqKsRZIhkl7A-7KYxuywOafo3Nd7L4BDA2NWhQoalyZdcNWp5NZAxq5DpSlP5dSPAhEGG2kXzMfUPTyyqXckY3gLp" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="907" data-original-width="1214" height="299" src="https://blogger.googleusercontent.com/img/a/AVvXsEhoAABmNJ04IBfwcuZF9dVH6ujw1BIckfTJydHLHJdSlZueCXIMc-ad7VzQ-APyWP93ISOE6XplO8QsfaA0948Q5PD7HMBclvQhqKsRZIhkl7A-7KYxuywOafo3Nd7L4BDA2NWhQoalyZdcNWp5NZAxq5DpSlP5dSPAhEGG2kXzMfUPTyyqXckY3gLp=w400-h299" width="400" /></a></div><br />This visualization is great for quickly browsing and or searching for specific items using the UI filters. For instance, you can check all your Cloud Storage buckets filtering them on the left pane:<p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEj32ewQFR7Z8v6PnTRdmMOrV27TS5WbeeQBXAf_LZ2feOo2KAYL-4-qzG7XAKW39q9RCTWEsOEFocYMLF2oAKJUtL1gzus-ETUuH-S_0ZXu8lP2uyXZK8XAo_Ae-Ls_0pVEl7_im279kEa7xvVtSvF2DJivUn-qv3jLlrHlb36zmsjAgmqmYEvVGdlu" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="897" data-original-width="1300" height="276" src="https://blogger.googleusercontent.com/img/a/AVvXsEj32ewQFR7Z8v6PnTRdmMOrV27TS5WbeeQBXAf_LZ2feOo2KAYL-4-qzG7XAKW39q9RCTWEsOEFocYMLF2oAKJUtL1gzus-ETUuH-S_0ZXu8lP2uyXZK8XAo_Ae-Ls_0pVEl7_im279kEa7xvVtSvF2DJivUn-qv3jLlrHlb36zmsjAgmqmYEvVGdlu=w400-h276" width="400" /></a></div><br />Doing this in the UI is cool and all but then you may need to run some more <b>complex filtering</b> and we may need to use a different approach. One of the cool things that I find on Google Cloud is that the almost all tasks you can do on the Cloud Console, usually can also be done both via REST APIs and via the <span style="font-family: courier;">gcloud</span> command line interface.<p></p><p>Let's say you need to export all the assets you have in Google Cloud for a spreadsheet to do a report to your management. First, we need grab our Organization numeric identifier. This can be done using the <a href="https://cloud.google.com/shell/docs" target="_blank">Cloud Shell</a> (a free VM Google gives you integrated in the Cloud Console):</p>
<pre class="prettyprint">gcloud organizations list</pre>
<p>This will print out some info, and we need to grab one of them for later use. Let's store it in an environment variable to use in the next commands:</p>
<pre class="prettyprint">export ORG_ID="$(gcloud organizations list --format='value(name)')"</pre>
<p>Now, let's export all resources underneath our organization with the Asset Inventory sub-command:</p>
<pre class="prettyprint">gcloud asset list \
--organization="$ORG_ID" \
--content-type=resource \
--format=json
</pre>
<p>This will print to the standard output of the terminal all your assets in JSON format. Now, you can already use the data for, say, do some shell-script foo like <span style="font-family: courier;">grep</span> commands to find specific items. But let's go a bit further with the <span style="font-family: courier;">gcloud</span> command line options.</p><p>First, you may need to export <b>specific types of assets</b>. This can be done using the <span style="font-family: courier;">--asset-types</span> parameter. You pass a coma separated list of asset types, like <span style="font-family: courier;">compute.googleapis.com/Instance,</span> to export only VM Instances, or <span style="font-family: courier;">storage.googleapis.com/Buckets</span> for Storage Buckets. Let's combine both to see all VMs and Buckets in our organization:</p>
<pre class="prettyprint">gcloud asset list \
--organization="$ORG_ID" \
--content-type=resource \
--asset-types='compute.googleapis.com/Instance,storage.googleapis.com/Bucket' \
--format=json</pre>
<p>This filtered the items but still shows up all the JSON attributes. Let's trim it a bit by projecting only a few attributes. For instance, let's export:</p><p></p><ul style="text-align: left;"><li>The asset type</li><li>The resource API endpoint</li><li>The resource location</li><li>The last update time</li></ul><p></p>
<pre class="prettyprint">gcloud asset list \
--organization="$ORG_ID" \
--content-type=resource \
--asset-types='compute.googleapis.com/Instance,storage.googleapis.com/Bucket' \
--format='csv(assetType, resource.data.selfLink, resource.data.location, updateTime)'</pre>
<p>Here we used the <span style="font-family: courier;">--format=csv()</span> to project the <b>desired columns</b> and print that into a convenient format to import somewhere else, like in a Google Sheet, for reporting purposes. Let's wrap it up directing the command output into a file:</p><pre class="prettyprint">gcloud asset list \
--organization="$ORG_ID" \
--content-type=resource \
--asset-types='compute.googleapis.com/Instance,storage.googleapis.com/Bucket' \
--format='csv(assetType, resource.location, updateTime, resource.data.selfLink)' \
> assets.csv</pre><p>Now, if you executed the command in Cloud Shell, you can use the Explorer to browse to it and then right click on the file name and download it, then upload it to a spreadsheet:</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEiC59lNdVCPWuYS2Fw3CD9N-HoHTU91tXsZvDl16o865tDj7eZN4E_3Y3U692dGPlClnIdSClgbIEAkPo9oUZL0CRTwp3KSwMzk7iBLVCL97H-vQSW-qyqTvG_OqbMBL0nn1N2RXWjAUK_Nd38WZ2FlOqmjoCvG3ztMNAX3TYxat5rZwYoe5dc1OHD4" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="676" data-original-width="943" height="286" src="https://blogger.googleusercontent.com/img/a/AVvXsEiC59lNdVCPWuYS2Fw3CD9N-HoHTU91tXsZvDl16o865tDj7eZN4E_3Y3U692dGPlClnIdSClgbIEAkPo9oUZL0CRTwp3KSwMzk7iBLVCL97H-vQSW-qyqTvG_OqbMBL0nn1N2RXWjAUK_Nd38WZ2FlOqmjoCvG3ztMNAX3TYxat5rZwYoe5dc1OHD4=w400-h286" width="400" /></a></div><br />What we discussed here for the Asset Inventory commands works also for other gcloud sub-commands. Check out <a href="https://cloud.google.com/sdk/docs/cheatsheet" target="_blank">this page</a> to learn more about all the nice things you can do with it!<p></p><p>Happy Hacking!</p>Ronoaldo Pereirahttp://www.blogger.com/profile/02620222149209242748noreply@blogger.com0São Paulo, SP, Brasil-23.5557714 -46.6395571-51.86600523617885 -81.79580709999999 4.7544624361788443 -11.483307099999998tag:blogger.com,1999:blog-6072471169748958029.post-45874330455607824322015-08-31T07:00:00.002-03:002022-05-16T02:51:39.905-03:00Migrating a Bitbucket Mercurial project to Git<div class="separator" style="clear: both; text-align: left;">I am a long-time Mercurial user. It was my first choice after the switch from Subversion to the DVCS world. The decision I took by that time was based on how simple Mercurial is. From the design of the filesystem, the friendly and simple command line options, to the fact that with a single instalation, I was able to have hg web serving the central repositories to the team.</div>
<br />
The time passed and I got used to Mercurial a lot. But looking around, seems that this is not the case for everyone else. Given the popularity of Github as a <i>coding social network</i>, the majority of services and tools has Git/Github support, if not both. Furthermore, Git has matured a lot, is very performant, and if the entire Linux Kernel uses it, why should I use anything else?<br />
<br />
After resisting a lot for the change, I finally settle that it was time to finally make it. Now I am happy with the switch, and slowling I am moving my projects, specially the Open Source ones, from Hg to Git. After using Git for a while, the <i>brain muscle memory </i>toke care of the confusing CLI options and switches. I still land on StackOverflow to find some switches, tought.<br />
<br />
Despite my change from Mercurial to Git, I'm probably going to stick with Bitbucket for some projects, specially the private ones and the ones at my corporation. The current pricing model of Bitbucket that charges for team members as opposed to repositories fits better in my monthly budget.<br />
<br />
In this post, we will discuss a few ways to move <u>from Mercurial to Git on Bitbucket</u>, migrating the whole project, not only the repository.<br />
<h3>
Rename the old repo and create a new one</h3>
The first step is to <i>backup</i> your Mercurial repository, by renaming it. It is wise to also remove any WRITE permissions to avoid people trying to push to the wrong repository during the migration if it is a shared one.<br />
<br />
To rename your repository go the project Settings page, then change the "Name" field to something different. In our case, I'll add the .hg suffix, so we can keep the same name for the new Git repository. Once renamed, you can then reuse the old name to create a new Git project.<div>
<br />
<h3>
Convert from Mercurial to Git</h3>
Now that the project has was renamed, let's convert the project commit history into a new, Git project. The best way to accomplish that is using the <i>hg-git</i> extension. This extension makes a great deal to interact with Git repositories from Mercurial itself, allowing you to simply push to the Git new path from a local copy.<br />
<br />
To install the extension, follow the instructions from <a href="http://hg-git.github.io/" target="_blank">their website</a>. If you are using Linux and have easy_install on your machine, it should be as simple as:<br />
<br />
<pre>$ easy_install --user hg-git</pre>
<br />
The <span style="font-family: "Courier New",Courier,monospace;">--user</span> flag tells easy_install to keep things on your home folder, making it easier to remove the extension later if you want to.<br />
<br />
After you have the extension installed, enable it in your <span style="font-family: "Courier New",Courier,monospace;">~/.hgrc</span> file:<br />
<br />
<pre>
[extensions]
...
hggit=
</pre>
<br />
Now, it is time to push the changes to the new repository. In order to avoid losing any commits, let's push from a clean, bare Mercurial repo. The process is as simple as cloning, creating a bookmark named master for the default branch, then pushing to the git repository:<br />
<pre>
$ hg clone ssh://hg@bitbucket.org/username/repository.hg && cd repository.hg
$ hg bookmark -r default master
$ hg push git+ssh://git@bitbucket.org/username/repository.git
</pre>
<br />
The <span style="font-family: "Courier New",Courier,monospace;">.hg</span> suffix on the first command is required only if you renamed the repository that way. On the other hand, the <span style="font-family: "Courier New",Courier,monospace;">.git</span> suffix on the Git repository name is <b>mandatory</b> so hg-git can make the push. The commands bellow assume that you have added your SSH key to the Bitbucket account.<br />
<br />
<b><i>Note</i></b>: we need to create a bookmark to the default version called <i>master</i>. This allows the hg-git extension to also create the Git master branch. If you don't do that step, you will end up with a repository that display no branches or commits on the Bitbucket site.<br />
<br />
The push URL <span style="font-family: "Courier New",Courier,monospace;">git+ssh</span> will convert the project using the hg-git extension first, then upload the contents to the Git repository specified. All changesets will be converted. It is important to notice that they will also all be recreated, generating new hashes based on the Git checksum.<br />
<h3>
Migrate Wiki</h3>
Bitbucket wikis are also repository tied to your project. However, the project wiki, once enabled, already has an initial commit. Because of that, there is no easy way to migrate the wiki history here, since hg-git will refuse to push to the new wiki repo since that wiki has a different ancestor. The easier way to migrate the wiki is to clone both the old wiki and the new one, and then copy the contents from the old folder to the new folder. Then you can push the changes to the new project wiki.<br />
<br />
<pre>
$ hg clone ssh://hg@bitbucket.org/username/repository.hg/wiki repository.hg-wiki
$ git clone git@bitbucket.org:username/repository.git/wiki repository.git-wiki
$ copy -r repository.hg-wiki repository.git-wiki
$ rm -r repository.git-wiki/.hg
$ cd repository.git-wiki && git push
</pre>
<h3>
Migrate Issues</h3>
If you also use the issue tracker, Bitbucket offers a nice export/import wizard to move them to the new repository. In the source project <i>Settings</i> page, go to <i>Import & Export</i> in the <i>Issues</i> menu. Then you can export the issues to a zip file, that you can later import to the new repository.<br />
<br />
Note here that changeset references in the issue tracker will all be broken: since the repository was converted, the changesets references inside issue comments now point to a broken link. One way to fix this is to parse the issue file format, that is a JSON array of issue objects, and rewrite the changesets within it. In my case, I didn't bother having the dead links for now, but I'll update this post if I ever find a good way to keep the references later.<br />
<h3>
Conclusion</h3>
With the three steps bellow, you can convert the repository from Mercurial to Git, keeping the majority of the data on the new repository. The caveats are that some issue comment links, as well as the changeset ids, will be different on the new repository.<br />
<br />
Happy Hacking!</div>Ronoaldo Pereirahttp://www.blogger.com/profile/02620222149209242748noreply@blogger.com0tag:blogger.com,1999:blog-6072471169748958029.post-49975654465544465412015-04-17T10:00:00.000-03:002015-04-17T10:00:07.091-03:00Backup to cloud when there is no space left on device<div class="separator" style="clear: both; text-align: center;">
<a href="https://docs.google.com/uc?id=0B9NjifSUullgZ083bWJiVXJyM00&export=download" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="258" src="https://docs.google.com/uc?id=0B9NjifSUullgZ083bWJiVXJyM00&export=download" width="400" /></a></div>
<br />
Imagine that you are planing to move some data from one server to another. To do so, you backup your data by creating a simple compressed <span style="font-family: Courier New, Courier, monospace;">tar</span> archive with:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">$ tar cjf /tmp/backup.tar.bz2 $HOME</span><br />
<br />
And, for your surprise you end up with a <span style="font-family: Courier New, Courier, monospace;">no space left on device</span> error from tar. You need to backup and have no more space: what do you do? Simple, use some Unix Pipes to the job!<br />
<h3>
gsutil</h3>
In my case, my source box already had the <a href="https://cloud.google.com/sdk" target="_blank">Google Cloud SDK</a> configured, and a good network connection to upload my backup. Instead of doublig the space locally and uploading later, what I did was to <i>pipe</i> the backup generated from tar directly to gsutil:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">$ tar cjf - . | gsutil cp - gs://mybucket/backup.tar.bz2</span><br />
<br />
<a href="http://en.wikipedia.org/wiki/Pipeline_%28Unix%29" target="_blank">Piping</a> is, basically, connecting the standard output of a process into the standard input of another one. This is, simply put, a way to take the output generated from <span style="font-family: Courier New, Courier, monospace;">tar</span> and sending it directly to <span style="font-family: Courier New, Courier, monospace;">gsutil</span>, which in turn will do a stream upload to the cloud.<br />
<br />
The same piping technique can be used with other tools as well. If you are copying from one server to another via SSH, you can also pipe it over with:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">$ tar cjf - . | ssh user@backupserver 'cat > backup.tar.gz'</span><br />
<br />
You can also use the similar technique to restore without the need to download everything.<br />
<br />
Happy hacking!<br />
<br />
--<br />
References<br />
[1] <a href="http://www.cyberciti.biz/faq/howto-use-tar-command-through-network-over-ssh-session/">http://www.cyberciti.biz/faq/howto-use-tar-command-through-network-over-ssh-session/</a><br />
[2] <a href="https://cloud.google.com/storage/docs/gsutil/commands/cp#streaming-transfers">https://cloud.google.com/storage/docs/gsutil/commands/cp#streaming-transfers</a><br />
[3] <a href="http://en.wikipedia.org/wiki/Pipeline_%28Unix%29">http://en.wikipedia.org/wiki/Pipeline_%28Unix%29</a>Ronoaldo Pereirahttp://www.blogger.com/profile/02620222149209242748noreply@blogger.com0tag:blogger.com,1999:blog-6072471169748958029.post-23681396903683486182015-04-05T00:21:00.001-03:002015-04-05T00:21:57.244-03:00Setting up a Moodle instance with Compute Engine and Cloud SQLThings are getting cloudy these days. And that's good! We have a lot of options available now, ranging from bare virtual machines, to fully managed platforms, and with all kinds of containerized environments in between.<br />
<br />
In this blog post, we are going to explore the Google Cloud Platform to host a Moodle site, using a distributed setup that's easy to build and scale as we need to upgrade. The setup we are going to build looks like this:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbcGSDVOJ8IOfAXex-RaXXoBqDLMWQJZuY4jamVCYY6_lsMXoDRn2wZ9KjmiD05JCrizx2aI0NxCQ7W01kvslX6hrTuxMmvjDaxS1r9Dk8wz1mYS9G9Mv84G_f14DQMF14u7mVEpMWto4/s1600/Moodle_google_cloud_layout.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbcGSDVOJ8IOfAXex-RaXXoBqDLMWQJZuY4jamVCYY6_lsMXoDRn2wZ9KjmiD05JCrizx2aI0NxCQ7W01kvslX6hrTuxMmvjDaxS1r9Dk8wz1mYS9G9Mv84G_f14DQMF14u7mVEpMWto4/s1600/Moodle_google_cloud_layout.png" height="253" width="320" /></a></div>
<ul>
<li>One Compute Engine Instance, where we will install Moodle software</li>
<li>One Persistent Disk attached to the instance in Read/Write mode, that we will use to store Moodle Data</li>
<li>One Cloud SQL Instance, to store the Moodle Database</li>
</ul>
<div>
This setup has some advantages over putting all into a single instance. First, it uses more than one node to host the system, distributing the load between the web-server and the database. This improves performance, and allows you to scale vertically, either by increasing the Cloud SQL instance tier, or by upgrading your VM to a more performant one. Secondly, we split software from data using both a separated database and a separated persistent disk. This aproach makes sofware updates easy to manage because the software is isolated from the data disk. Making backups with this setup is also very easy: we can enable the Cloud SQL backups to keep up to 7 backups, and also create a very simple script to backup our Moodle Data regularly using Diferential Snapshots on the Persistent Disk.</div>
<h3>
Getting Started</h3>
<div>
The first thing to do is to sign up to Google Cloud Platform, using a Google Account. You can use the USD $300 free-trial credit to get started! The free trial credit is valid for 60 days, and it is only valid for new accounts.</div>
<div>
<br /></div>
<div>
Sign up at <a href="https://cloud.google.com/free-trial/" target="_blank">https://cloud.google.com/free-trial/</a>. <i>Note: in order to use the free-trial, or to use the Compute Engine virtual machines, you need to setup a valid payment method, and may need to put in your credit card</i>. To use the free-trial, you will create a new project; if you already use Cloud Platform, I also recomend you to create a new project to host only your Moodle system, unless you plan to share other resources with it.</div>
<div>
<br /></div>
<div>
With your account setup, you can start by creating a new Compute Engine VM. Compute Engine is billed by the minute, after the first 15 minutes, making it easy to setup, stop then resume the work if you need to. Let's begin with a <span style="font-family: Courier New, Courier, monospace;">g1-small</span><span style="font-family: inherit;"> (1 vCPU, 1.7 GB memory) </span>instance, that can handle a small to moderated traffic very well, and is cost effective.</div>
<h3>
Launch and configure the virtual machine</h3>
<div>
<span style="font-family: inherit;">To launch the instance from the Developers Console, click on your project, then navigate to <i>Compute -> Compute Engine -> VM Instances</i>. In the Instances panel, click on the <i>New Instance</i> button.</span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://docs.google.com/uc?id=0B9NjifSUullgV3U0YnNCcXRTaFU&export=download" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="266" src="https://docs.google.com/uc?id=0B9NjifSUullgV3U0YnNCcXRTaFU&export=download" width="400" /></a></div>
<div>
<br /></div>
<div>
In the new instance screen, let's create our instance with an extra disk to store Moodle data. To do so, click on the <i>Show advanced options</i> link, and use the following values:</div>
<div>
<ul>
<li>Instance name: moodle-server</li>
<li>Firewall: Allow HTTP traffic</li>
<li>Zone: choose one that apply for your location</li>
<li>Machine Type: g1-small (you can choose a higher one if you expect more traffic)</li>
<li>Boot Disk: new from image, and Debian wheezy backports as boot disk image option</li>
<li>Under additional disks, create a new one by clicking on the plus sign, and choose create a new disk.</li>
<ul>
<li>Disk name: moodle-data</li>
<li>Disk type: regular persistent disk should go just fine, but if you expect a high load, I suggest SSD as they are more performant</li>
<li>Source type: none</li>
<li>Size: 200G. You can choose lower/higher values here, but keep in mind that this will be used by Moodle data, so it requires a reasonable ammount of space, and that the higher your disk, more performance it has</li>
<li>Click on create (the disk is created immediately)</li>
</ul>
<li>In networking, I recommend you to reserve a static IP: just click on <i>New static IP</i> and give it a name (moodle-ip). The IP is reserved as you click on this option</li>
<li>For the authentication options, I suggest you choosing Read/Write to Compute and Storage, and Enable to Cloud SQL. This allows you to run scripts to backup data from within your instance</li>
<li>Finally, click on the <i>Create</i> button</li>
</ul>
<div>
Your instance will be started, and we are going to be able to work with it as soon as it boots up. This process is usually very fast, and in one or two minutes the instance will show up in the VM Instances screen. To manage the instance, we need to login using SSH. You can use the SSH button from the Instances page:</div>
</div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://docs.google.com/uc?id=0B9NjifSUullgS1FHSlB0N3NVSkk&export=download" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="248" src="https://docs.google.com/uc?id=0B9NjifSUullgS1FHSlB0N3NVSkk&export=download" width="400" /></a></div>
<div>
The web-based shell will open in a new window, and a new user account will be added to your instance. Note that the web-based shell can get disconnected and to avoid loosing your work or corrupting your instance, let's use a terminal multiplexer that will keep the shell running if the SSH session is droped. I suggest <span style="font-family: Courier New, Courier, monospace;">tmux</span>, as it is very easy and straghtforward to install/run:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">sudo bash</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">apt-get update && apt-get upgrade --yes</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">apt-get install tmux --yes</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">tmux</span></div>
<div>
<br /></div>
<div>
Once running, tmux will show a green bar at the bottom of the screen:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://docs.google.com/uc?id=0B9NjifSUullgd0g3SXM2SzNlV2M&export=download" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="243" src="https://docs.google.com/uc?id=0B9NjifSUullgd0g3SXM2SzNlV2M&export=download" width="400" /></a></div>
</div>
<div>
<br /></div>
<div>
To leave tmux, use CTRL+D, to detach it and keep it running, or <span style="font-family: Courier New, Courier, monospace;">exit</span> to close the shell session. If your connection is dropped, reopen the ssh window and reattach to tmux using:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">sudo </span><span style="font-family: 'Courier New', Courier, monospace;">tmux attach-session -t 0</span></div>
<h4>
Format and mount the disk</h4>
<div>
Remember that we have attached a empty disk to the VM? Since it is empty, we need to format it with a filesystem, and mount it on a directory. To do so, we use a program shipped by Google called <span style="font-family: Courier New, Courier, monospace;">safe_format_and_mount</span>, that does just that: format the disk only if it is not formated and mount it for you.</div>
<div>
<br /></div>
<div>
First, let's create a mount point:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">mkdir -p /mnt/moodledata</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: inherit;">Then let's format the disk (<i>line breaks added for clarity, this is a single command without the back slashes</i>):</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">/usr/share/google/safe_format_and_mount \</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> -m "mkfs.ext4 -F" \</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> /dev/disk/by-id/google-moodle-data /mnt/moodledata</span></div>
<div>
<br /></div>
<div>
Check that the device is mounted listing the contets:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">ls -l /mnt/moodledata</span></div>
<div>
<br /></div>
<div>
You should see a <span style="font-family: Courier New, Courier, monospace;">lost+found</span> entry, for an empty Ext4 filesystem. To make this mount point be remounted during boot, add one entry to the <span style="font-family: Courier New, Courier, monospace;">/etc/fstab</span> file:</div>
<div>
<br /></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;">cat >> /etc/fstab</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;">/dev/disk/by-id/google-moodle-data /mnt/moodledata defaults 1 1</span></div>
<div>
<br /></div>
<div>
<b><i>Important! </i></b>Use double grather than signs (<span style="font-family: 'Courier New', Courier, monospace;">>></span>), otherwise you will loose the root mount point from the fstab file!</div>
<h3>
Launch and configure the Database</h3>
<div>
Now, let's launch the database using Cloud SQL. To do that, navigate to <i>Storage -> Cloud SQL</i> menu, then click on the <i>New instance</i> button:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://docs.google.com/uc?id=0B9NjifSUullgLXRQNHdvTDdfMkk&export=download" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="328" src="https://docs.google.com/uc?id=0B9NjifSUullgLXRQNHdvTDdfMkk&export=download" width="400" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
Fill the form with the appropriate values:</div>
<div>
<ul>
<li>Instance name: moodle-db</li>
<li>Region: Choose the same geographic region of you Compute Engine VM</li>
<li>Choose the D0 tier. You can safely start with D0, and upgrade later if you see high usage on the database</li>
<li>Click on the <i>Show advanced options</i> link</li>
<li>Choose the MySQL 5.5 version</li>
<li>Choose a billing plan. In this case, as Moodle will constantly talk to your database, and since you will need an static IP address, the per use plan may not make much difference. I recommend choosing the daily package plan</li>
<li>Select the same region of you compute engine VM to the location option</li>
<li>Let <i>Enable backups</i> option checked and choose a more suitable time frame for backups; this can be changed later</li>
<li>Check the assign an IPv4 address to the instance. Altought a free IPv6 address is available, we can't use this one from within Compute Engine yet</li>
<li>Under allowed networks, click on the + sign, and put the IP address of your compute engine VM; you can put the same name of your vm: moodle-server</li>
<li>Click on the <i>Create</i> button to launch the instance</li>
</ul>
<div>
The server instance starts very fast. We need to create a new MySQL user and a new database for Moodle. To do so, click on the instance name in the list, and then on the <i>Databases</i> tab, then on the <i>New database</i> button:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://docs.google.com/uc?id=0B9NjifSUullgbFBhbWIwWUhxZWc&export=download" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://docs.google.com/uc?id=0B9NjifSUullgbFBhbWIwWUhxZWc&export=download" /></a></div>
<div>
<br /></div>
<div>
In the new database form:</div>
</div>
<div>
<ul>
<li>Database name: moodle</li>
<li>Character set: utf-8</li>
<li>Collation: default collation</li>
<li>Click on the <i>Add</i> button.</li>
</ul>
<div>
To create the user, click the <i>Access Control</i> tab, then on the <i>Users</i> tab, and finally click on the <i>New User</i> button.</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://docs.google.com/uc?id=0B9NjifSUullgZ0NSNlBwNnJjUVU&export=download" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://docs.google.com/uc?id=0B9NjifSUullgZ0NSNlBwNnJjUVU&export=download" /></a></div>
<div>
<br /></div>
<div>
In the new user form:</div>
</div>
<div>
<ul>
<li>Username: moodle</li>
<li>Password: create a strong password, and take note of it to use on the Moodle setup</li>
<li>Click on the <i>Add</i> button</li>
</ul>
<div>
Now we are ready to install and configure our Moodle server! Let's get back to our compute engine VM terminal window.</div>
</div>
<h3>
Installing and configuring Moodle software</h3>
<div>
Moodle requires a web server and a few PHP modules. You can choose the webserver you know best, and for this tutorial I'll use the Apache webserver. Since this is a Debian server, let's use <i>apt</i> to download and install everything we can, so we benefit from the Debian Security upgrades:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">apt-get install apache2 php5 php5-gd php5-mysql php5-curl php5-xmlrpc php5-intl</span></div>
<div>
<br /></div>
<div>
In order to get the Moodle software, let's choose the latest stable version from the download site: <a href="https://download.moodle.org/releases/latest/">https://download.moodle.org/releases/latest/</a>. Click on the <i>Download tgz</i> button, and dismiss the automatic download window: we need the direct download link. Right click on the <i>click here for manual download</i> link and copy the link address:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://docs.google.com/uc?id=0B9NjifSUullgdEtwY09WQnBGb3M&export=download" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="202" src="https://docs.google.com/uc?id=0B9NjifSUullgdEtwY09WQnBGb3M&export=download" width="400" /></a></div>
<div>
<br class="Apple-interchange-newline" />On the terminal window, we need to download into our instance using <i>curl</i>. We can type <span style="font-family: Courier New, Courier, monospace;">curl</span> and then paste the copied link with CTRL+V, then finish the command with <span style="font-family: Courier New, Courier, monospace;">> moodle.tar.gz</span>, resulting in a command like this one:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">curl https://download.moodle.org/download.php/direct/stable28/moodle-latest-28.tgz > moodle.tar.gz</span></div>
<div>
<br /></div>
<div>
<i><b>Important</b></i>: do not forget the "<span style="font-family: Courier New, Courier, monospace;">> moodle.tar.gz</span>" part of the command at the end, otherwise your file will be <i>printed</i> on the screen! Before unpacking the moodle software, remove the default index.html page from Apache:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">rm /var/www/index.html</span></div>
<div>
<br /></div>
<div>
Then, use the tar command to unpack the software in the Apache directory:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">tar xf moodle.tar.gz -C /var/www --strip-components=1</span></div>
<div>
<br /></div>
<div>
The <span style="font-family: Courier New, Courier, monospace;">tar</span> command bellow extracts the file contents into <span style="font-family: Courier New, Courier, monospace;">/var/www</span>, skipping the creation of the <span style="font-family: Courier New, Courier, monospace;">moodle</span> sub-directory. Now we need to configure Moodle to use all the resources we launched.</div>
<div>
<br /></div>
<div>
Moodle will need read/write permissions on the software and data directories. To fix permissions on these places, use the <i>chown</i> command:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">chwon www-data:www-data -R /var/www</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">chwon www-data:www-data </span><span style="font-family: 'Courier New', Courier, monospace;">/mnt/moodledata</span></div>
<div>
<br /></div>
<div>
To configure Moodle, launch the setup wizard by navigating to your instance IP, or by clicking on the compute engine VM IP link on the VM instances page:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://docs.google.com/uc?id=0B9NjifSUullgS1pLbzh2NkFEMVk&export=download" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="188" src="https://docs.google.com/uc?id=0B9NjifSUullgS1pLbzh2NkFEMVk&export=download" width="400" /></a></div>
<div>
You should see the Moodle setup page:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://docs.google.com/uc?id=0B9NjifSUullgR0s3SWRLbWJDN0k&export=download" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="238" src="https://docs.google.com/uc?id=0B9NjifSUullgR0s3SWRLbWJDN0k&export=download" width="400" /></a></div>
<div>
<br /></div>
<div>
Select yor language, then click on the <i>Next</i> button. Moodle requires you to set some important options. The first one is the visible website address; let's keep the instance IP for now, and we change this value later when the setup is finished. You also need to tell Moodle the software directory and the data directory. The software directory is where we extracted Moodle, <span style="font-family: Courier New, Courier, monospace;">/var/www</span>, and the data directory will be the mount point we have created with the 200G persistent disk: <span style="font-family: Courier New, Courier, monospace;">/mnt/moodledata</span>. Put these values and click <i>next</i>.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://docs.google.com/uc?id=0B9NjifSUullgdU5YQ1ZWa3RRcGc&export=download" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="213" src="https://docs.google.com/uc?id=0B9NjifSUullgdU5YQ1ZWa3RRcGc&export=download" width="400" /></a></div>
<div>
<br /></div>
<div>
In the next page, we need to setup the database connection. First, select the <span style="font-family: Courier New, Courier, monospace;">mysqli</span> database driver, then click <i>next</i>. In the database connection settings fill the values:</div>
<div>
<ul>
<li>Database host: the IP address of your Cloud SQL instance</li>
<li>Database name: moodle</li>
<li>Database username: moodle</li>
<li>Database password: the password you choose previously</li>
<li>Tables prefix: optional, just use the default</li>
<li>Leave empty database port and unix socket, as we are going to use the defaults</li>
</ul>
<div>
When you click <i>next</i>, Moodle will connect to your instance, and it will start in response for the first connection. Also, Moodle will present the software license; read it, and if you agree, click <i>Continue</i> to install.</div>
<div>
<br /></div>
<div>
In the next screen, if some PHP modules that are required, they will be listed in red. You can install them and hit reload to let the installer check again. If all is OK, just click on the <i>Continue</i> button to proceed with the installation.</div>
</div>
<div>
<br /></div>
<div>
The next step will take a reasonable time: Moodle installer will configure all database tables, populate some initial data, prepare the <span style="font-family: Courier New, Courier, monospace;">moodledata</span> directory with some cache values, and get all things in place. Wait until the page finish loading, and you see a <i>Continue</i> button.</div>
<div>
<br /></div>
<div>
After all setup is completed, you can create the Admin user. The admin user is the one with access to all setup pages, and you should choose a strong password for it. After you create the admin user, you will be propted to put some basic site information. Fill the form and click <i>Save changes</i>.</div>
<div>
<br /></div>
<div>
Now you should see your Moodle home page, and you can start playing around, creating users and courses, installing plugins and so forth.</div>
<h3>
Enable the cron job</h3>
<div>
Moodle itself requires you to configure a job that runs periodically, called a cron job. The easier way to setup a Moodle cron job is by running it using <i>curl</i>. Let's enable the moodle cron with a simple shell script and the Debian run-parts directory for cron jobs. First, create a simple shell script that fetches the cron url, and logs the output for debugging:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">cat > /etc/cron.hourly/moodle-cron</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">#!/bin/sh</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">curl http://localhost/admin/cron.php 2>&1 > /var/log/moodle-cron.log</span></div>
<div>
<br /></div>
<div>
To finish the file, type CTRL+D and check the contents with the cat command:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">cat /etc/cron.hourly/moodle-cron</span></div>
<div>
<br /></div>
<div>
Now, let's make it executable, and run a test cron:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">chmod +x /etc/cron.hourly/moodle-cron</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">/etc/cron.hourly/moodle-cron</span></div>
<div>
<br /></div>
<div>
The cron job will run and you can check it's output at the log file.</div>
<h3>
Sending e-mail</h3>
<div>
Compute Engine VMs are not allowed to send e-mail. However, you can configure Moodle to use an SMTP server hosted on another service. Google partned with Sendgrid to deliver your compute engine emails, so you can follow <a href="https://cloud.google.com/compute/docs/tutorials/sending-mail" target="_blank">this page</a> to signup to a free account that can send 25000 emails from your Google projects. Once signed up, grab the Sendgrid credentials and configure the hostname, port 2525, username and password using the Moodle <a href="https://docs.moodle.org/28/en/Messaging_settings#SMTP_hosts" target="_blank">e-mail setup instructions</a>. Alternatively, you can setup Mandrill, that also has a free quota for emails per month, and provides an SMTP server connection just like Sendgrid.</div>
<h3>
Improvements</h3>
<div>
Some improvements that can be done:</div>
<div>
<ul>
<li>Add another daily cron script that backup the moodle-data disk, as well as the moodle-server disk (the boot disk). See <a href="https://cloud.google.com/compute/docs/disks/persistent-disks#snapshots" target="_blank">how to take disk snapshots</a> for backups, and take care to make your script remove old backps at the end.</li>
<li>Change the site URL in the config.php. Moodle saves some data in the database with the full URL, so you should use the <a href="https://docs.moodle.org/28/en/Search_and_replace_tool" target="_blank">database search and replace tool</a> to change data to the new URL.</li>
</ul>
</div>
<h3>
Conclusion</h3>
<div>
In this post, we learned how to setup a simple, efficient and cost-effective Moodle site, that can handle a mid-size Moodle traffic, and provides a good infra-structure that can be vertically upgraded in the future.</div>
<div>
<br /></div>
<div>
<i><b>Important!</b> You will be billed by the resources used, so if you did this as a test-only, remember to remove the compute engine VM, the persistent disks, and the Cloud SQL instance.</i></div>
Ronoaldo Pereirahttp://www.blogger.com/profile/02620222149209242748noreply@blogger.com0tag:blogger.com,1999:blog-6072471169748958029.post-51054484759414780732015-03-31T19:00:00.000-03:002015-03-31T20:56:24.312-03:00Building custom Linux kernel the Debian way<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiaKKnHIFnq-h8WttfImZeAeKWAkv3NdONxrSz5YQyQCLY5Nkpz7uxO6XuTAJZfu0iqHNNeWouaAVBfntaS2ozO8cAkhz8rJ3QcabO4jn2got7aKQ0i7lKrVFmwsu56gCP46t8NhsfmFMo/s1600/terminal-window.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiaKKnHIFnq-h8WttfImZeAeKWAkv3NdONxrSz5YQyQCLY5Nkpz7uxO6XuTAJZfu0iqHNNeWouaAVBfntaS2ozO8cAkhz8rJ3QcabO4jn2got7aKQ0i7lKrVFmwsu56gCP46t8NhsfmFMo/s1600/terminal-window.png" height="316" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Image by: <a href="https://openclipart.org/user-detail/shokunin" target="_blank">shokunin</a></td></tr>
</tbody></table>
Building a Linux kernel is a nice way to learn some new skills, install a fresh and new version of your Operating System core software, and also make any optimizations you see fit for your machine. However, building a kernel is not a trivial task. From downloading, configuring, building and installing, you may get lost if you don't follow some tips.<br />
<br />
In this blog post, we are going to understand some basic steps on how to build a custom kernel from sources, using some tools available in the <a href="https://www.debian.org/" target="_blank">Debian GNU/Linux</a> operating system. If you use any other Debian derivative, like <a href="http://www.ubuntu.com/" target="_blank">Ubuntu</a> or <a href="http://www.linuxmint.com/" target="_blank">Mint</a>, this tutorial should work as well.<br />
<h3>
Why build a custom Kernel?</h3>
First and most important: because you can! Secondly: because it is fun! And, off course, because you may need a newer kernel to support hardware, or may need to rebuild the kernel to enable specific features.<br />
<br />
It is also a good experience that allows you to hack into the biggest open source project, see all available configuration options and disable those that you may never use, making a slim and optimized kernel for your machine.<br />
<h3>
Understanding the process</h3>
In Debian and derivatives, all packages are installed by the <i>dpkg</i> tool, or via the <i>apt</i> command-line and it's friends. These ensures that every file installed by your package is tracked by the system, allowing you to properly upgrade or remove it later. If you install something in Debian without using <i>dpkg/apt</i>, then you risk compromising your system and any future upgrades from Debian.<br />
<br />
Building a Debian package is a complex task by itself: you need to follow a set of rules in order to do it right. But for some commonly used software, Debian also provides a few tools that help you automate the process of generating a .deb installable file out of the source tarball. This is true for the JDK, as well as for the Kernel.<br />
<br />
To build a kernel from sources and generate a Debian packge using the tools available in the repository, you need to install the make-kpkg package:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"># apt-get install make-kpkg build-essential</span><br />
<br />
The process to build the kernel is:<br />
<ol>
<li>Download the sources from <a href="http://www.kernel.org/">http://www.kernel.org/</a></li>
<li>Extract the source tarball: </li>
<ul>
<li><span style="font-family: 'Courier New', Courier, monospace;">tar xf linux-3.19.3.tar.xz</span></li>
</ul>
<li>Configure the Kernel options enabling/disabling features:</li>
<ul>
<li><span style="font-family: 'Courier New', Courier, monospace;">cd linux-3.19.3 && make menuconfig</span></li>
</ul>
<li>Compile all sources to generate the kernel binary and modules:</li>
<ul>
<li><span style="font-family: 'Courier New', Courier, monospace;">make-kpkg --revision=~custom1 -j 4 binary_image</span></li>
</ul>
</ol>
<div>
This process is simple, except from the configuration options: if you don't know how to configure the kernel, you will get lost with the <span style="font-family: Courier New, Courier, monospace;">make menuconfig</span> script. Since configuring the kernel itself is a hard thing to properly understand and master, let's use another script to to that in a more automated way. Instead of <b><span style="font-family: Courier New, Courier, monospace;">make menuconfig</span>, use: <span style="font-family: Courier New, Courier, monospace;">make olddefconfig</span></b>. This will take the current running kernel options as a base, and update it to match the new kernel options, using the default values for new options.</div>
<h3>
Helper script</h3>
<div>
This process can be even further automated using a script I wrote called <a href="https://bitbucket.org/ronoaldo/tools/src/tip/bin/?at=default" target="_blank">kernelbuild</a>. The script is available under the set of scripts I host at the <i>tools</i> repository on my Bitbucket account. To build the newest stable kernel version, all you need to do is:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">$ kernelbuild --fast --safe-config</span></div>
<div>
<br /></div>
<div>
That's it. The script downloads the latest stable kernel from www.kernel.org, unpacks the source in the current directory, runs a simple option to automatically configure the kernel based on your current running one and uses auomated defaults for newer build options, and then compiles the kernel using the make-kpkg tool. This process will generate all <span style="font-family: Courier New, Courier, monospace;">.deb</span> files for you: the kernel image, a linux-headers package and a package with debug symbols. Super easy!</div>
<h3>
How the script works</h3>
<div>
The script uses <span style="font-family: Courier New, Courier, monospace;">wget</span> to download the kernel package for you. It parses the <a href="http://kernel.org/">kernel.org</a> HTML output to find the latest stable version, and downloads that version tarball for you. You can avoid autodetecting the kernel version to download a specific one with the <span style="font-family: Courier New, Courier, monospace;">-k</span> switch.</div>
<div>
<br /></div>
<div>
Once downloaded, the script unpacks the source tarball. After that step, the script runs the <span style="font-family: Courier New, Courier, monospace;">make olddefconfig</span> kernel configuration script. Altought simple, this can be odd if you are using a very different kernel release than the one you are building. You may be prompted to set some options by hand, and if unsure, just press ENTER to use the default value.</div>
<div>
<br /></div>
<div>
The next step is to build the kernel, and the script passes a few parameters to <span style="font-family: Courier New, Courier, monospace;">make-kpkg</span>, such as a custom version tag (<span style="font-family: Courier New, Courier, monospace;">~custom1</span>). This version number uses a tilde, so your kernel package may be upgraded by the oficial Debian kernel release once it is available in the repositories. The script also uses some other build options, like the <span style="font-family: Courier New, Courier, monospace;">-j</span> flag that enables parallell build and use all <a href="https://plus.google.com/+RonoaldoPereira/posts/WDrJfUwDmza" target="_blank">CPU cores available</a>.</div>
<h3>
Installing your new Kernel</h3>
<div>
To install your debian kernel, all you need to do is to use the <i>dpkg</i> tool. Use ls to see the deb files built, and install the image and header packages. If you built the 3.19.3 kernel version, to install use:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># dpkg -i linux-image-3.19.3_3.19.3~custom1_amd64.deb</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># dpkg -i linux-headers-3.19.3_3.19.3~custom1_amd64.deb</span></div>
<div>
<br /></div>
<div>
This installs the packages built by the script, and you can then reboot your machine to use the new kernel. The <span style="font-family: Courier New, Courier, monospace;">make-kpkg</span> tool builds a package that contains all triggers that will update/generate the <span style="font-family: Courier New, Courier, monospace;">initrd</span> file (necessary to boot), and appropriate Grub menu entries.<br />
<h3>
Conclusion</h3>
</div>
<div>
Building the Linux Kernel is a very interesting task, that can help you better understand how an operating system works and learn more about the options available to optimize and customize your machine software.</div>
<div>
<br /></div>
<div>
In this post, we saw some tips to build a Kernel suitable to use on a Debian-based distribution, and how you can make use of a helper script to generate a Debian package that integrates nicelly with the operating system software.</div>
<div>
<br /></div>
<div>
<i>Disclaimer: I have tested the scripts and commands provided in this post on multiple machines without issues. The source code is provided as is, without any warranties, and I am not resposible for any damange that you cause to your software or hardware when using it. Use at your own risk.</i></div>
<div>
<i><br /></i></div>
<div>
Happy hacking!</div>
Ronoaldo Pereirahttp://www.blogger.com/profile/02620222149209242748noreply@blogger.com0tag:blogger.com,1999:blog-6072471169748958029.post-16704058742863820812015-03-25T00:09:00.001-03:002015-03-25T10:37:39.307-03:00Multiple runtimes into single Google App Engine project with ModulesThere is no single tool to rule them all. Each <a href="https://cloud.google.com/appengine/" target="_blank">Google App Engine</a> runtime has it strengths, but sometimes you can't afford to reinvent the wheel and reimplement something that is ready to use in a library, because of the language you choose for a project in the first place. With the App Engine <a href="https://cloud.google.com/appengine/docs/python/modules/" target="_blank">Modules</a> feature, you can mix and match different programming languages that share the same data in a single <a href="https://cloud.google.com/" target="_blank">Google Cloud Platform</a> project.<br />
<br />
In this post, we discuss what the modules feature is and how to leverage it's power with a sample application that mixes Go and Python.<br />
<br />
For the impatient, here is the source code <a href="https://github.com/ronoaldo/appengine-multiple-runtimes-sample">https://github.com/ronoaldo/appengine-multiple-runtimes-sample</a>.<br />
<h3>
What are Modules?</h3>
Google Cloud Platform has a top-level organization unit called Project. Each project comes <i>pre-configured</i> with an App Engine application. By creating a project, you have access to a Cloud Datastore database, Memcache, and Task queue, all ready to use.<br />
<br />
In the early days of App Engine, your app could have multiple versions, but there was no option to configure <i>backend</i> machines. Once the now deprecated Backends configuration was released, there was no way to deploy multiple versions of them.<br />
<br />
With the Modules feature, you can <i>split</i> your application and have a different set of configurations for performance, scaling and version under the same Project, one for each module. This allows you to mix different runtimes, but still share the same state in the same project: all modules have access to the same resources, such as Datastore and Memcache. This allows you to, for instance, create a backend in Java that generates reports with the App Engine Map/Reduce, while you leverage Go in the frontend, also called the default module, that has auto-scaling.<br />
<h3>
That's just what I want! How to use this stuff?</h3>
You can have any setup you want depending on your codebase. The Modules API just requires you to specify the <i>default</i> module, and any other number of <i>non-default</i> modules when running and deploying.<br />
<br />
You specify the module name using the <span style="font-family: Courier New, Courier, monospace;">module:</span> directive in <i>app.yaml</i> and the <span style="font-family: Courier New, Courier, monospace;"><module></span> XML node in the <i>appengine-web.xml</i> configuration files. A file that does not contain any module directive, means that it is the default module; you can also explicitly set <span style="font-family: Courier New, Courier, monospace;">module: default</span>, which I recommend, to avoid mistakes. To create other modules, all you need is to configure their names on separate configuration files. For Java, you can also use the standard project layout of an EAR (Java Enterprise Application) and set the configuration options under each application folder (each module).<br />
<br />
My suggested project layout is one that has separate one folder for each module. For instance, in our sample application, we are going to have this layout:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">project/</span><br />
<span style="font-family: Courier New, Courier, monospace;"> go-frontend/</span><br />
<span style="font-family: Courier New, Courier, monospace;"> app.yaml</span><br />
<span style="font-family: Courier New, Courier, monospace;"> main.go</span><br />
<span style="font-family: Courier New, Courier, monospace;"> py-backend/</span><br />
<span style="font-family: Courier New, Courier, monospace;"> app.yaml</span><br />
<span style="font-family: Courier New, Courier, monospace;"> main.py</span><br />
<br />
In the go-frontend folder, we put our App Engine app in Go. Then, in the py-backend folder we put our backend logic in Python. To make all this works you have to:<br />
<br />
<ol>
<li>Make sure that go-frontend/app.yaml has the <span style="font-family: Courier New, Courier, monospace;">module: default</span><span style="font-family: inherit;"> directive.</span></li>
<li>Make sure that py-backend/app.yaml has the <span style="font-family: Courier New, Courier, monospace;">module: py-backend</span> (or any other name you want, except the word <i>default</i>)</li>
<li>Make sure that both app.yaml files have the same <span style="font-family: Courier New, Courier, monospace;">application:</span> element value.</li>
<li>Use the <span style="font-family: Courier New, Courier, monospace;">gcloud</span> command to install the App Engine SDKs.</li>
</ol>
<div>
With that setup, you can then run the app locally using the command:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">$ gcloud preview app run go-backend py-frontend</span></div>
<div>
<br /></div>
<div>
The gcloud preview app subcommand will then parse all files you have and launch them locally. The default module run on port 8080, and the py-backend on 8081, so you can address them by the port number.</div>
<div>
<br /></div>
<div>
To deploy, you can use:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">$ gcloud preview app deploy go-backend py-frontend</span></div>
<div>
<br /></div>
<div>
With this setup, you have some options:</div>
<div>
<ol>
<li>Each module can live on separated source code repositories, making easy to manage their lifecycle independently from each other.</li>
<li>You can deploy a single module by just putting only one folder on the command line.</li>
</ol>
<div>
Some tips:</div>
</div>
<div>
<ol>
<li>Remember that <b>all modules share the same <i>state</i>, but not share code</b>. That said, be careful when changing the datastore. For instance, if you have a struct in Go that saved more properties than you Python db.Model has, <b>you may lose data</b>! In this particular case, it would be nice to use something like a raw <b>db.Expando (Python) or a datastore.PropertyList (Go) to manipulate the data on the non-default module</b> or to <i>always</i> keep your models in sync between the codebases.</li>
<li>When deploying, you may need to <i><b>convert your app performance settings to modules</b></i>. That means you no longer can change instance class on the Application Settings, and instead, you need to specify this in the app.yaml / appengine-web.xml files. (See the modules <a href="https://cloud.google.com/appengine/docs/python/modules/converting" target="_blank">documentation on how to migrate</a>).</li>
<li>This scenario is particularly useful to run App Engine Pipelines or App Engine Map/Reduce jobs using the Python runtime backend while keeping the front-end in Go.</li>
<li><b>Some settings are global to your appliation like the task queue, cron, and dispatch entry definitions</b>. So keep these files, <span style="font-family: Courier New, Courier, monospace;">queue.yaml</span>, <span style="font-family: Courier New, Courier, monospace;">cron.yaml,</span> and <span style="font-family: Courier New, Courier, monospace;">dispatch.yaml</span>, <b>only on the default module</b>. If you have three queues defined in the go-frontend folder and only one in the py-backend, then if you deploy only the py-backend, only one queue will be on the server, and this can break things, particularly for the cron job definitions.</li>
</ol>
<h3>
If I have custom domains, how can I make this work?</h3>
<div>
You can use the address wildcards to access the backends. Let's say you have a wildcard mapping of you custom domain like this:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">*.mydomain.com CNAME glehosted.com</span></div>
<div>
<br /></div>
<div>
App Engine will then route request as follows:</div>
<div>
<br /></div>
<div>
<b>www.mydomain.com</b> is handled by the default version of go-frontend</div>
<div>
- since no module is named www, and no version subdomain was specified, this falls into the default module, default version</div>
<div>
<br /></div>
<div>
<b>default.mydomain.com</b> is handled by the default version of go-frontend</div>
<div>
- matches the <i>default</i> module/default version</div>
<div>
<br /></div>
<div>
<b>py-backend.mydomain.com</b> is handled by the default version of py-backend</div>
<div>
- matches the <i>py-backend</i> module/default version</div>
<div>
<br /></div>
<div>
<b>beta.py-backend.mydomain.com</b> is handled by the version beta if it exists, of py-backend</div>
<div>
- matches the <i>py-backend</i> module/<i>beta</i> version</div>
<div>
<br /></div>
<div>
Any string that does not match a module is handled by the default module, and any string that does not match a version, is handled by the default version of the module. This is the same for *.your-app-id.appspot.com domain that you have for free in your app.</div>
<div>
<br /></div>
<div>
You can also configure some rules that change that behavior using the <span style="font-family: Courier New, Courier, monospace;">dispatch.yaml (dispatch.xml)</span> file. This file allows you to specify wildcard URLs and say from which modules they will be served. For instance, you can map */report/* to route requests to any hostname, that have a path starting with /report/, to be served to your Python backend.</div>
<div>
<br /></div>
<div>
Note that, if you dispatch /report/* to the python backend, the python backend handlers will handle requests with that path prefix (/report/), and not at the root path. Therefore, you need to map you WSGI app using that path as a prefix. See more about routing requests to modules in <a href="https://cloud.google.com/appengine/docs/python/modules/routing" target="_blank">their docs</a>. Check out the sample code dispatch.yaml as well.</div>
<h3>
Conclusion</h3>
</div>
<div>
With Modules, your App Engine applications can be composed by a more robust infrastructure, allowing you to mix different programming languages and runtimes. If you work with micro services on App Engine, this is a great fit to better take advantage of your app resources using the right language and tools to do the right job.<br />
<br />
--<br />
Update 25/03/2015 - Fixed grammar errors, and added clarifications to first paragraphs.</div>
Ronoaldo Pereirahttp://www.blogger.com/profile/02620222149209242748noreply@blogger.com0tag:blogger.com,1999:blog-6072471169748958029.post-24827350191445606992013-12-30T00:53:00.002-02:002013-12-30T00:53:38.637-02:00Tutorial: How to get started with Apache Cordova and Firefox OS on Nitrous.IOIt's time to go <i>mobile</i>. And it is time to go <i>cloud</i>. Why not <i>go mobile in the cloud</i>? In this tutorial we'll learn how get started building mobile web apps using Apache Cordova (formelly PhoneGap) and testing your app using the Firefox OS Simulator. To avoid some setup complexity and to allow for coding entirelly in the cloud, let's build the app using a Node.js Nitrous.IO box.<br />
<br />
<h3>
1 - Signup for Nitrous.IO and spawn a Node.js box</h3>
<a href="http://devblog.ronoaldo.net/2013/08/coding-in-cloud-part-i-nitrousio-review.html" target="_blank">Nitrous.IO</a> is cloud development environment, that allows you to quick and easy setup a linux box in the cloud with zero-install: just using your browser. If you don't have a Nitrous account yet, you can signup for one on <a href="https://www.nitrous.io/join/2Mbr_lb9O3M" target="_blank">https://www.nitrous.io/</a>, using Google, Github, or by using a username and password.<br />
<br />
Once logged into Nitrous, start a new box, and choose the Node.js box template:<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://docs.google.com/uc?id=0B9NjifSUullgRk1SLVVQaWVKckU&export=download" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="225" src="https://docs.google.com/uc?id=0B9NjifSUullgRk1SLVVQaWVKckU&export=download" width="400" /></a></div>
<br />
The IDE will open after your box is ready, with the following image:<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://docs.google.com/uc?id=0B9NjifSUullgbDZfZmc2N216Z1U&export=download" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="285" src="https://docs.google.com/uc?id=0B9NjifSUullgbDZfZmc2N216Z1U&export=download" width="400" /></a></div>
The black screen in the bottom is the Console window, where you will type the commands.<br />
<br />
<h3>
2 - Install Apache Cordova CLI using npm</h3>
<a href="http://cordova.apache.org/" target="_blank">Apache Cordova</a> is a mobile application container, that allows you to build Web Apps that can be installed on mobile devices, and benefit from the device features, such as Camera or GPS. I recomend reading the documentation to get a grasp on it: at least <a href="http://cordova.apache.org/docs/en/3.3.0/guide_overview_index.md.html#Overview" target="_blank">Overview</a> and <a href="http://cordova.apache.org/docs/en/3.3.0/guide_cli_index.md.html#The%20Command-Line%20Interface" target="_blank">Command Line Interface</a> (or the Web Project Dev).<br />
<br />
To install cordova on your box, run the command below in the console window:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">$ npm -g install cordova</span><br />
<br />
NPM is the Node Package Manager, and the Cordova CLI is distributed as a node module. The above command will download and install the Cordova CLI tools and install them on your Nitrous box, under the ~/.nvm folder. This may take some minutes, and when it the instalation is complete, you can test it with the command<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">$ cordova info</span><br />
<br />
This command will display some details about how to use cordova, and about your cordova instalation.<br />
<br />
<h3>
3 - Start a new cordova project</h3>
With Cordova installed, we are ready to start the project itself. In your console window, run the commands:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">$ cd ~/workspace</span><br />
<span style="font-family: Courier New, Courier, monospace;">$ cordova create hello</span><br />
<br />
This will create a project under <span style="font-family: Courier New, Courier, monospace;">~/workspace/hello</span>, and you will be able to see some folders:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">$ ls hello/</span><br />
<span style="font-family: Courier New, Courier, monospace;">merges platforms plugins www</span><br />
<br />
The folders <span style="font-family: Courier New, Courier, monospace;">merges</span>, <span style="font-family: Courier New, Courier, monospace;">platforms</span> and <span style="font-family: Courier New, Courier, monospace;">plugins</span> will be empty, but we will fill some of them soon. The <span style="font-family: Courier New, Courier, monospace;">merges</span> folder hosts platform specific content overrides. The <span style="font-family: Courier New, Courier, monospace;">platforms</span> folder will host the cordova.js JavaScript bindings, plus some configurations and other media. The <span style="font-family: Courier New, Courier, monospace;">plugins</span> folder will host additional plugins, as you add them.<br />
<br />
The <span style="font-family: Courier New, Courier, monospace;">www</span> folder is your application root. Cordova will create a small sample app that responds to the 'deviceready' event in that folder. The index.html file references the app script, style sheet and image assets.<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://docs.google.com/uc?id=0B9NjifSUullgcEwtem5fSURvQlk&export=download" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://docs.google.com/uc?id=0B9NjifSUullgcEwtem5fSURvQlk&export=download" width="214" /></a></div>
<br />
<h3>
4 - Add the Firefox OS platform</h3>
To make your app deployable, Cordova needs some platform specific JavaScript bindings, plus the target platform SDK to build the final app. You can add as many platforms as you wish, with the command <span style="font-family: Courier New, Courier, monospace;">cordova platrorm add <i><platform shortcut></i></span>. Some platforms, like <span style="font-family: Courier New, Courier, monospace;">ios</span> (from Apple) or <span style="font-family: Courier New, Courier, monospace;">wp8</span> (from Microsoft), require a particular operating system to build and run your app on emulators (see details <a href="http://cordova.apache.org/docs/en/3.3.0/guide_cli_index.md.html#The%20Command-Line%20Interface_prerequisites" target="_blank">here</a>). In this tutorial, we'll be using the <span style="font-family: Courier New, Courier, monospace;">firefoxos</span> (from Mozilla) platform, since it can be tested entirelly using the web.<br />
<br />
To add the Firefox OS platform, run the command, from the hello folder<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">$ cd ~/workspace/hello</span><br />
<span style="font-family: Courier New, Courier, monospace;">$ cordova platform add firefoxos</span><br />
<br />
After that command finishes, the platforms folder will have a new sub-folder, with the cordova bindings and some sample configurations. We are ready to test our app now!<br />
<br />
<h3>
5 - Test in the Nitrous.IO browser</h3>
Cordova can serve your app from a built-in web server, so you can test your preview the app on your browser.<br />
<br />
Once we have added a platform, we can run:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"><b>$ cordova serve 3000</b></span><br />
<span style="font-family: Courier New, Courier, monospace;"> Generating config.xml from defaults for platform "firefoxos"</span><br />
<span style="font-family: Courier New, Courier, monospace;"> Preparing firefoxos project</span><br />
<span style="font-family: Courier New, Courier, monospace;"> Static file server running on port 3000 (i.e. <b>http://localhost:3000</b>)</span><br />
<span style="font-family: Courier New, Courier, monospace;"> CTRL + C to shut down</span><br />
<br />
Nitrous allow you to do live testing of servers started from your box. Click on the <i>Preview menu -> Port 3000</i>, and a new browser window will open, displaying the Cordova serve main page:<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://docs.google.com/uc?id=0B9NjifSUullgdE5id0x3Q3hiQlk&export=download" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://docs.google.com/uc?id=0B9NjifSUullgdE5id0x3Q3hiQlk&export=download" width="192" /></a></div>
<br />
This page lists some app info, and the platforms and plugins available. Note that the <span style="font-family: Courier New, Courier, monospace;">firefoxos</span> platform is a link you can click and, voila! Your app is running!<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://docs.google.com/uc?id=0B9NjifSUullgek9NSFd2QlJRdzg&export=download" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="316" src="https://docs.google.com/uc?id=0B9NjifSUullgek9NSFd2QlJRdzg&export=download" width="400" /></a></div>
<br />
<br />
The same URL can be used to access the app on your mobile phone or tabled, using the device browser, so you can preview how it may look like in the smaller screen.<br />
<br />
<h3>
6 - Install the Firefox OS Simulator</h3>
Besides using a tools that allow you to preview the web page on mobile device iframes, you can also use a real device emulator for this task: the Firefox OS Simulator. The device simulator is a Firefox add-on that allows you to test web apps designed to Firefox OS. To install it, launch Firefox (or Iceweasel if you're on Debian), and go to the Firefox OS Simulator <a href="https://addons.mozilla.org/en-US/firefox/addon/firefox-os-simulator/" target="_blank">add-on page</a>, then click "Add to Firefox" button. The download and setup may take some time, the add-on has more than 60MB.<br />
<br />
Once installed, the simulator can be launched from the Firefox (Iceweasel) menu -> Developer Tools -> Firefox OS Simulator.<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://docs.google.com/uc?id=0B9NjifSUullgdnNXeHNNNG9lbFU&export=download" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="260" src="https://docs.google.com/uc?id=0B9NjifSUullgdnNXeHNNNG9lbFU&export=download" width="400" /></a></div>
<br />
<h3>
7 - Run your app in the simulator</h3>
The simulator will allow you to add a URL for your app. Copy the link of your app preview and paste it to the Firefox OS Simulator URL bar, then click on the "Add URL" button:<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://docs.google.com/uc?id=0B9NjifSUullgc3VBYUxseE1jYjg&export=download" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="61" src="https://docs.google.com/uc?id=0B9NjifSUullgc3VBYUxseE1jYjg&export=download" width="320" /></a></div>
<br />
The device simulator will launch, download and install your app for testing. And you will see the app running on the simulator:<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://docs.google.com/uc?id=0B9NjifSUullgd250ZmlaMHJMWVU&export=download" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="191" src="https://docs.google.com/uc?id=0B9NjifSUullgd250ZmlaMHJMWVU&export=download" width="400" /></a></div>
<br />
The simulator is a pretty nice, and give you an idea on how your app fits on a real device. You can test the device with your mouse, and navigate over the OS and test launching and closing the app.<br />
<br />
<h3>
Conclusion</h3>
The web of today is the web accesible from Mobile to 4K TVs. The mobile market is growing fast, and the web is here to help you jump in. Using technologies such as Cordova (PhoneGap) you can reach a broader audience of mobile customers, by using well known web standards. Cloud Computing enables infinite scaling and computational power, helping you build awesome apps that can do amazing things. Nitrous helps you get started on almost <i>everything</i>, by providing a simple and easy to use web interface, with a powerfull GNU/Linux box in the cloud.<br />
<br />
<h3>
References</h3>
<br />
<ul>
<li><a href="http://cordova.apache.org/" target="_blank">Apache Cordova</a> - Apache Software Foundation</li>
<li><a href="https://developer.mozilla.org/en-US/Apps/Quickstart/Build/Your_first_app#Tools_.26_Testing" target="_blank">Firefox OS: Your First App</a> - Mozilla Foundation</li>
<li><a href="https://developer.mozilla.org/en-US/docs/Tools/Firefox_OS_Simulator" target="_blank">Firefox OS Simulator</a></li>
</ul>
<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">/happy hacking</span><br />
<br />Ronoaldo Pereirahttp://www.blogger.com/profile/02620222149209242748noreply@blogger.com0tag:blogger.com,1999:blog-6072471169748958029.post-63506613137794995972013-11-16T21:24:00.002-02:002013-11-16T21:24:35.426-02:00Coding in the Cloud - Part II - Google Apps Script and Google Sites ReviewContinuing the Coding in the Cloud series, today we'll take a look at the possibilities to quick and easy build web sites and smart web applications using some cool tools from Google: Google Sites and Google Apps Script.<br />
<div>
<br /></div>
<div>
<a href="https://sites.google.com/" target="_blank">Google Sites</a> is a tool to help you quick and easy build a website. You can think of it as a CMS tool, with some goodies: built-in themes, mobile optimization, site search, customization of theme and layout, and a few other nice integrations with other Google Services. You can, for instance, embedd a Google Drive Form into your site with a few mouse clicks.<br />
<br />
<a href="https://developers.google.com/apps-script/" target="_blank">Apps Script</a> is a JavaScript rutime that runs in the Cloud, and that allows you to extend and automate the Google Apps suite, improving Google Docs, Forms, Spreadsheets and Sites. It has a lot of builtin support for a Google APIs, allowing you to integrate your script with both Google Apps and Google Cloud services.<br />
<h2>
Development Envionment</h2>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgNNLWNmhrUP8vVOPVGCsOZht8aN07MF8iOYoXAW1NovSK07pLvZEwuHjCtZkUt7j_JzTYC_1VBBTCsTcDjhJ9cqi1VvNowJZ69EJxjeMSUm6FVF4XxUEHMPXX4W4vi8XRzYmWLXVLOK_E/s1600/google-apps-script.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="247" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgNNLWNmhrUP8vVOPVGCsOZht8aN07MF8iOYoXAW1NovSK07pLvZEwuHjCtZkUt7j_JzTYC_1VBBTCsTcDjhJ9cqi1VvNowJZ69EJxjeMSUm6FVF4XxUEHMPXX4W4vi8XRzYmWLXVLOK_E/s400/google-apps-script.png" width="400" /></a></div>
<div>
<br /></div>
With your standard Google Account or with a Google Apps Account, you have access to the Google Drive and, from there, to the Apps Script Editor. From Google Drive, you can create standalone scripts that you can publish either as libraries or web applications. You can also create scripts from containers, like a Spreadsheet, a Document, a Form or a Site.<br />
<br />
The Script editor has some nice features. It runs in your browser, has syntax higlight for HTML, JavaScript and CSS, and also features an integrated debugger. From the editor, you can manage the files within your project, that can be Script files (with .gs extension), and HTML files (with .html extension).<br />
<h2>
Web Apps, Widgets and Web Services</h2>
With Apps Script you can build apps, widgets for Sites or Docs, and even simple web services. Your script can handle the HTTP GET and POST methods with callbacks, and serve content like HTML, XML, JSON and many more using the HtmlService and the ContentService modules. You can also build usefull widgets and embedd them into Sites or other Google Drive apps.<br />
<h2>
Import and Export features</h2>
A recently launched feature that is very usefull is the ability to <a href="https://developers.google.com/apps-script/import-export" target="_blank">import and export standalone scripts</a>. You can, for instance, create a github project page for a Apps Script library, and manage your source versions from there. Once you have it done, you can then publish it to Google Drive and create a new deployed version from there.<br />
<br />
Taking advatage of this resource, a few weeks ago the Google Plugin for Eclipse received an update <a href="http://googleappsdeveloper.blogspot.com.br/2013/10/total-eclipse-of-apps-script.html" target="_blank">allowing you to edit your Apps Script code within Eclipse</a>, and sync back to the Cloud.<br />
<h2>
Easy integration with other Google APIs</h2>
</div>
<div>
With the seamless integration with other APIs, you can build awesome apps. You can use services like Big Query, Google Prediction, Google Cloud SQL, and many other Cloud APIs, appart from nice integration with Google Drive, Gmail and other Google Apps services. All services expose JavaScript libraries to help you get started and use take advantage of a lot of funcionality from those tools.<br />
<br />
A few time ago, I flooded my inbox with unwanted, repeated e-mails from a bad mail alert on one of my systems. It was about 5 milion e-mails, impossible even to let Gmail alone to remove. Thanks to Apps Script, I was able to write a simple, batch removal, capable to cleanup my inbox.</div>
<h2>
Triggers</h2>
<div>
With container-bound and non-container bound triggers, you can schedule your script to run daily, or in response of an event, like a Form submission. A good use case, is to send e-mail alerts every day, on a simple help-desk like Spreadsheet, reminding users from the open tickets.</div>
<h2>
Conclusion</h2>
<div>
Initially designed to be a way to users customize and extend funcionality from Google Apps services, Apps Script is upgrading himself to a very nice, and easy to use tool. Using the Google Sites CMS features with the Apps Script programming in the backend, you can quickly build a web app that range from simple corporative tools to full featured e-commerce web sites. Seamless integration with the Cloud and other Google Services allow your app to be built and deployed entirely in the Google Cloud!</div>
<div>
<br /></div>
<div>
Stay tunned to the next post in the series!</div>
Ronoaldo Pereirahttp://www.blogger.com/profile/02620222149209242748noreply@blogger.com0tag:blogger.com,1999:blog-6072471169748958029.post-21806174405793199602013-09-04T19:50:00.001-03:002013-11-16T21:27:02.409-02:00Mapping multiple domains to Google AppEngine with nginx reverse-proxy on GCEIf you work with Google AppEngine and want to map a custom domain to your application, you must also configure Google Apps for that domain. If you have an app with a white label option, your customers also have to configure Google Apps in order to use your app under their domain. Not only that, they must also manage the Google Apps setup, outside your application.<br />
<br />
There are two issues on the public issue tracker that you may be interested to star and watch. One about creating an API to manage the domains: <a href="https://code.google.com/p/googleappengine/issues/detail?id=8528">https://code.google.com/p/googleappengine/issues/detail?id=8528</a> and another about the problem itself: <a href="https://code.google.com/p/googleappengine/issues/detail?id=2587">https://code.google.com/p/googleappengine/issues/detail?id=2587</a>. I recommend both to be starred so we tell to Google AppEngine Team that this is important!<br />
<br />
But if you simply can't wait for them to be fixed, you can fix it yourself building your own <i>reverse-proxy</i> solution, and make it scale with your app. With a simple nginx proxy setup, running on a GCE instance, a round-robin DNS mapping and a small set of changes into your source code, you may acomplish that with very little effort. As a bonus, those proxies can be usefull for other stuff too, like another level of caching that can save you some instance hits.<br />
<h2>
Initial setup</h2>
<div>
First of all, I'm assuming that you already have an AppEngine application with billing enabled, and that you have created a Google Cloud Console project for your application. If you don't have this already, then sign up for AppEngine, setup billing on your app and setup billing on the Google Cloud Console for you application. This will allow you to use all resources required.<br />
<br />
If your AppEngine application don't have a Cloud Console Project yet, go to you AppEngine Dashboard -> Application Settings -> Cloud Integration and click in the button "Add Project". After that procedure take place, you will see a box like this one:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOozbhyphenhyphenoqUs0yBIcmQjvhRss3gIAn1yjMWPpvSakt_3Kpx4e02s8dVGVKF6R_HsZ9_O6c2iuWAQTH2-k6azNsVg6jF96ezcNkiAluzOBWBA5vFrBwAp6_9gwXKX-fxRbABvilI2B7jrG8/s1600/nginx-proxy-multiple-domains_img01.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOozbhyphenhyphenoqUs0yBIcmQjvhRss3gIAn1yjMWPpvSakt_3Kpx4e02s8dVGVKF6R_HsZ9_O6c2iuWAQTH2-k6azNsVg6jF96ezcNkiAluzOBWBA5vFrBwAp6_9gwXKX-fxRbABvilI2B7jrG8/s1600/nginx-proxy-multiple-domains_img01.png" /></a></div>
<div>
After that, login on Google Cloud Console at https://cloud.google.com/console, select the project with your application and go to Compute Engine -> VM Instances and click on the "New Instance" button. I recomend using the Debian Wheezy 7.0 image as starting point, as it is the current long-term stable release of Debian, and because Debian is the new default image to use in GCE.</div>
<div>
<br /></div>
<div>
In the new instance screen, give it a name and description. Also, I recomend label it with tags to help you manage more than one instance at once. I've added the tags <span style="font-family: Courier New, Courier, monospace;">nginx</span>, <span style="font-family: Courier New, Courier, monospace;">proxy</span> and <span style="font-family: Courier New, Courier, monospace;">http</span>. Later, we will use some tags to configure routes and firewalls.</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgUgzBDaa7uypGyVI-09mvKQ5FqlqMmWt3fFoLFX9E-ixI-fncXW9y5iB62D2NplSvt_S0D2S2aNPYjFMwFHZbia0VJzMKLmMKJjTB1WEeRy1W3NIcSIZZhIb2-KQCADRm08P36xoPjZ-A/s1600/nginx-proxy-multiple-domains_img03_gce_debian.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgUgzBDaa7uypGyVI-09mvKQ5FqlqMmWt3fFoLFX9E-ixI-fncXW9y5iB62D2NplSvt_S0D2S2aNPYjFMwFHZbia0VJzMKLmMKJjTB1WEeRy1W3NIcSIZZhIb2-KQCADRm08P36xoPjZ-A/s1600/nginx-proxy-multiple-domains_img03_gce_debian.png" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
It is also recommended that you choose "New static ip address..." in the Networking section, wich may allow you to configure the proxy more easily.</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiIz_vxJNf8jpngtdegsoD_e9gzPK3ifLEY6yMKawaquMKMhfkndQumDqbEoyutq4us3EFYTjAvfUeyYFxcqFa82SLymJWv7afcNuouDmWcTSWFs0c2zUvdpd7gCNzVyeIDq6rtkQZpkJI/s1600/nginx-proxy-multiple-domains_img02_gce_network.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiIz_vxJNf8jpngtdegsoD_e9gzPK3ifLEY6yMKawaquMKMhfkndQumDqbEoyutq4us3EFYTjAvfUeyYFxcqFa82SLymJWv7afcNuouDmWcTSWFs0c2zUvdpd7gCNzVyeIDq6rtkQZpkJI/s1600/nginx-proxy-multiple-domains_img02_gce_network.png" /></a></div>
<div>
<br /></div>
<div>
To finish your instance startup, click in the "Create" button and wait until your instance is ready.</div>
<h2>
Installing and configuring nginx</h2>
<div>
Once your instance boots, you are ready to configure the software that will allow it to proxy requests to your AppEngine application. I choose nginx as the software, because it is a lightweight and high scalable HTTP server. You can also do this with Apache or even Squid if you wish, but nginx scales easier and the configuration is very simple.</div>
<div>
<br /></div>
<div>
Login into your new instance via SSH, and issue the following commands:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">sudo bash</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">apt-get update && apt-get upgrade</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">apt-get install nginx unattended-upgrades</span></div>
<div>
<br /></div>
<div>
The first command starts a new bash session as root, as we will need to configure and install everything as super user. The second line updates package lists and install security upgrades. The last line installs our HTTP server, nginx, with default configuration and the package <a href="http://packages.debian.org/search?keywords=unattended-upgrades" target="_blank">unattended-upgrades</a>, to automatically install security patches.</div>
<div>
<br /></div>
<div>
Next, let's configure nginx to proxy requests from www.example.com to our application. AppEngine has some <a href="https://developers.google.com/appengine/docs/domain#more_about_wildcard_subdomain_mapping">wildcard domains</a> in place for every app. Any requests reaching *.default.<i>your-app-id</i>.appspot.com will reach the default version of your application. With that in mind, we can easily map www.example.com on nginx to proxy www.example.com.default.your-app-id.appspot.com.<br />
<br />
To put this into our newly installed nginx, let's create a file named /etc/nginx/conf.d/appspot.conf</div>
<div>
<br />
<b><span style="font-family: Courier New, Courier, monospace;">Listing 1: /etc/nginx/conf.d/appspot.conf</span></b><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">server {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> listen 80;</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> location / {</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"> resolver 8.8.8.8;</span></div>
<div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"> </span><span style="font-family: 'Courier New', Courier, monospace;"> </span><span style="font-family: Courier New, Courier, monospace;">proxy_set_header X-Forwarded-Host $host;</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"> </span><span style="font-family: 'Courier New', Courier, monospace;"> </span><span style="font-family: Courier New, Courier, monospace;">proxy_set_header X-Forwarded-Server $host;</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"> </span><span style="font-family: 'Courier New', Courier, monospace;"> </span><span style="font-family: Courier New, Courier, monospace;">proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"> </span><span style="font-family: 'Courier New', Courier, monospace;"> </span><span style="font-family: Courier New, Courier, monospace;">proxy_set_header X-Real-IP $remote_addr;</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"> </span><span style="font-family: 'Courier New', Courier, monospace;"> </span><span style="font-family: Courier New, Courier, monospace;">proxy_pass http://$host.default.<i>appid</i>.appspot.com/$request_uri;</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> proxy_redirect http://$host.default.<i>appid</i>.appspot.com/ /;</span></div>
</div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">}</span></div>
<div>
<br /></div>
<div>
Let's check this configuration:<br />
<br />
<ul>
<li>First, we create a "nginx server" at port 80</li>
<li>In that server, we set a resolver to the Google DNS 8.8.8.8</li>
<li>Then, we configure the proxy to add some headers. I'm setting all of them, to be safe, but they may not all be needed.</li>
<li>Finally, the proxy_pass and proxy_redirect directives will make requests be routed to your app, and when you return a 302 redirect it will hide the proxy internal name.</li>
</ul>
</div>
<h2>
Configure instance firewalls</h2>
<div>
To make your proxy available to the public, let's setup a firewall rule in the instance network to allow traffic incomming from any host to the port 80.<br />
<br />
In the Compute Engine Console, click on the VM Instances section, and then click on your recently launched instance. Under the instance details page, you'll see the Network section, where you can find a link to the newtork your instance is attached to. Click on the link and in the network configuration. under Firewall, and fill the form with the following values. I used the tag http to apply this setting to all instances with that tag. Very handy!<br />
<br />
<i><b>Tip</b>: for better control over your instances, you can create a new network under the Network section of GCE console, and then on instance creation, select the network to keep your proxies with the same setup.</i><br />
<div>
<h2>
Gotchas</h2>
</div>
<div>
As you may expect, there is aways a trade-off! This setup may cause some side-effects:</div>
<div>
<ol>
<li>UserService.createLoginUrl() aways redirect the user to the proxied address under appspot.com.</li>
<li>To your app, the remote address will aways be the proxy IP and the request hostname will be under appid.appspot.com.</li>
</ol>
<div>
You can solve #1 by replacing the login management to tools like <a href="https://code.google.com/p/openid4java/source/browse/#svn%2Ftrunk%2Fsamples%2Fappengine-consumer">openid4java</a>, or something similar, achieving the same goal. #2 is discussed bellow, and also require some code changes.</div>
</div>
</div>
<h2>
Update your code when you need the real hostname</h2>
<div>
You may already have some sort of code to make the hostname from the request map to your customer data and resources, like customizing the logo or a stylesheet. In this case, you have to tweak your code to handle the request headers we set on nginx instead of using only the values from the current request alone.<br />
<br />
If your web framework allows you to add some middlewere classes like Django, or a servlet Filter like Java EE, then you can use this to intercept requests, check for proxy headers, and wrap the request methods that return the hostname or request URL to use the values from the proxy. See bellow for a sample Java Filter that will do the trick (Listing 2):</div>
<div>
<br />
<b><span style="font-family: Courier New, Courier, monospace;">Listing 2: ProxyFilter.java</span></b><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">import java.io.IOException;</span><br />
<span style="font-family: Courier New, Courier, monospace;">import javax.servlet.*;</span><br />
<span style="font-family: Courier New, Courier, monospace;">import javax.servlet.http.*;</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">public class ProxyFilter implements Filter {</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"> public static final String HEADER = "x-forwarded-server";</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"> public class ProxyRequestWrapper extends HttpServletRequestWrapper {</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"> public ProxyRequestWrapper(HttpServletRequest request) {</span><br />
<span style="font-family: Courier New, Courier, monospace;"> super(request);</span><br />
<span style="font-family: Courier New, Courier, monospace;"> }</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"> @Override</span><br />
<span style="font-family: Courier New, Courier, monospace;"> public String getServerName() {</span><br />
<span style="font-family: Courier New, Courier, monospace;"> return getHeader(HEADER);</span><br />
<span style="font-family: Courier New, Courier, monospace;"> }</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"> @Override</span><br />
<span style="font-family: Courier New, Courier, monospace;"> public StringBuffer getRequestURL() {</span><br />
<span style="font-family: Courier New, Courier, monospace;"> StringBuffer url = new StringBuffer(getScheme());</span><br />
<span style="font-family: Courier New, Courier, monospace;"> url.append("://");</span><br />
<span style="font-family: Courier New, Courier, monospace;"> url.append(getServerName()); </span><br />
<span style="font-family: Courier New, Courier, monospace;"> url.append(":").append(getServerPort());</span><br />
<span style="font-family: Courier New, Courier, monospace;"> url.append(getRequestURI()); </span><br />
<span style="font-family: Courier New, Courier, monospace;"> return url;</span><br />
<span style="font-family: Courier New, Courier, monospace;"> }</span><br />
<span style="font-family: Courier New, Courier, monospace;"> }</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"> @Override</span><br />
<span style="font-family: Courier New, Courier, monospace;"> public void doFilter(ServletRequest proxyRequest,</span><br />
<span style="font-family: Courier New, Courier, monospace;"> ServletResponse proxyResponse,</span><br />
<span style="font-family: Courier New, Courier, monospace;"> FilterChain chain) throws IOException, ServletException {</span><br />
<span style="font-family: Courier New, Courier, monospace;"> HttpServletRequest request = (HttpServletRequest) proxyRequest;</span><br />
<span style="font-family: Courier New, Courier, monospace;"> </span><br />
<span style="font-family: Courier New, Courier, monospace;"> String header = request.getHeader(HEADER);</span><br />
<span style="font-family: Courier New, Courier, monospace;"> if (header != null && !header.trim().isEmpty()) {</span><br />
<span style="font-family: Courier New, Courier, monospace;"> request = new ProxyRequestWrapper(request);</span><br />
<span style="font-family: Courier New, Courier, monospace;"> }</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"> chain.doFilter(request, proxyResponse);</span><br />
<span style="font-family: Courier New, Courier, monospace;"> }</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"> @Override</span><br />
<span style="font-family: Courier New, Courier, monospace;"> public void init(FilterConfig config) {}</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"> @Override</span><br />
<span style="font-family: Courier New, Courier, monospace;"> public void destroy() {}</span><br />
<span style="font-family: Courier New, Courier, monospace;">}</span><br />
<br /></div>
<div>
You can make similar changes using a Django middleware. Please refer to the <a href="https://docs.djangoproject.com/en/dev/topics/http/middleware/">django documentation</a> for details.</div>
<h2>
Scaling your reverse-proxies </h2>
<div>
To scale your proxies, you can add more instances with another reserved IPs, and use DNS Round Robin as simple way to achieve load-ballancing.<br />
<br />
You just need to add multiple A records for a sudomain like <i>lb.myapp.com</i>, one for each proxy IP. Then, when mapping the example.com domain to your app, your customers can simply create a CNAME record to <i>lb.myapp.com.</i> The requests will be distributed over your proxies using the Round Robin algorithm. Please refer to your DNS hosting documentation for details, and to check if they provide this DNS algorithm.<br />
<br />
<i><b>Tip</b>: Since this nginx setup is very simple, you can configure a small startup script, host it on the Google Cloud Storage, and then make it easier to start new proxies: just call the command gcutil with the startup script URL as a parameter! After your new instance boots, just add the new reserved IP as a new A record for lb.myapp.com. This is not an automatic scaling, but is very easy to start/stop them, and also to remove nodes from your system.</i></div>
<div>
<h2>
Conclusion</h2>
</div>
<div>
This procedure can also be used on other cloud providers, like Amazon AWS or Digital Ocean, or any other provider with Deiban Wheezy support. This configuratoin steps can also work on other distributions or nginx versions, altought I didn't tested.</div>
<div>
<br /></div>
<div>
Hope you enjoy this tips, and if you run into problems, drop a comment bellow and we can help you out!</div>
<div>
<br /></div>
<div>
Happy Hacking!</div>
Ronoaldo Pereirahttp://www.blogger.com/profile/02620222149209242748noreply@blogger.com2tag:blogger.com,1999:blog-6072471169748958029.post-6783837853351871212013-08-07T09:08:00.007-03:002013-09-14T17:46:01.599-03:00Coding in the Cloud - Part I - Nitrous.IO ReviewHello fellows, this is the first post in the new Developer Blog by Ronoaldo. This blog post is also the first of a series of reviews on Cloud IDEs. They give you the power of <i>scaling</i> your development environment as the same way you scale your applications. Also, it is the same workspace on all your devices: no need to setup your workspace over and over again.<br />
<h2>
Code, test, build and deploy in the Cloud</h2>
<div>
A cloud development environment allow applications to be entirely coded, tested, built and deployed in the cloud. The concept of a Cloud IDE is gettig more and more attention, and this is the first post of a series of reviews about some of the tools I've tried.</div>
<div>
<br /></div>
<div>
I'll start the series with <a href="https://www.nitrous.io/join/2Mbr_lb9O3M" target="_blank">Nitrous.IO</a>. It has a clean design, extremelly fast responsive UI, comes bundled with all tools to jump in and start coding, allows you to run servers and other tools on your <i>box</i> and test you app live from there. It has all features that are really usefull to replace your local environment, and new features are comming.</div>
<h2>
It simply works</h2>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhJq-mc1_B9pes80ODbEb3gro1rou1aeqeyDUls591Nxxb1a4Oah2HNboFVPgNmCDDt-AW_zpFi3QGuFE2t3Fl9B87YoT83yH0NJOESoKSbKT897_-gB2LwHnh5RQdC7bR5euJS2yJPIFM/s1600/nitrous-io-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="279" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhJq-mc1_B9pes80ODbEb3gro1rou1aeqeyDUls591Nxxb1a4Oah2HNboFVPgNmCDDt-AW_zpFi3QGuFE2t3Fl9B87YoT83yH0NJOESoKSbKT897_-gB2LwHnh5RQdC7bR5euJS2yJPIFM/s400/nitrous-io-1.png" width="400" /></a></div>
<div>
<br /></div>
<div>
If you're used to run Eclipse locally, and try Nitrous.IO for Java, you'll probably miss a few of buttons, short cuts, menu entries and so one. I found that the key strengh that Nitrous has is that all things simply work. Period. I've tried other IDEs that has some fancy buttons, but they don't work for real worls projects. This gives you the felling that programming in the cloud will not work at all. <a href="https://www.nitrous.io/join/2Mbr_lb9O3M" target="_blank">Nitrous</a>, on the other hand, has just the small subset of tools that you need to develop online, and all features there simply work.</div>
<h2>
Fast user experience</h2>
<div>
I'm a big fan of running stuff on the terminal: it is simple to remember, easy to automate, and it gives you a full power of doing, well, anything! The terminal emulator of the <a href="https://www.nitrous.io/join/2Mbr_lb9O3M" target="_blank">Nitrous</a> IDE is one of the most amazing I've used. It allows you to fire up Vim and have the feelling that you're on your local computer. It responds so fast that you don't even realize that evey character you type goes over the internet. It is even faster than most SSH sessions I've being running over some cloud providers, with not freezing of the UI.</div>
<div>
<br /></div>
<div>
The IDE also has a clean and usefull code editor. The editor is very simple, has syntax highlight but no code completion. It comes with a dark theme, which helps you keep focus on your code. It also handles tabs/soft-tabs very nicely.</div>
<div>
<br /></div>
<div>
The possibility to invite people for your workspace is also cool, and works fine. You can edit the same file, doing a "pair programming session", with a remote co-worker. Wen you finish, you can simply remove access for your workspace. You have full control, unlike other tools where you can't remove someone you've added.</div>
<h2>
Flexibility</h2>
<div>
Another key strengh of <a href="https://www.nitrous.io/join/2Mbr_lb9O3M" rel="" target="_blank">Nitrous</a> is flexibility. The concept of "build your own plan" is really cool. My opinion is that this is something that all other players on the market didn't realized yet: every developer and project has different needs. There is no plan that will fits all, and bringing the "pay as you go" to IDE was a smart move. This opens a whole new gate of possibilities. It allows you to try out with less cost, and when you feel that the tool is ready to do the job, you just buy more NO2 and the thing runs faster or with more storage. You choose. Having choice to configure your workspace the way you need it is really important, specially if you work with different projects.</div>
<div>
<br /></div>
<div>
But the customization don't stop there: you can download and install on your box other toosl you'll need, making your box the perfect workspace for all your projects.</div>
<h2>
Fun</h2>
<div>
The whole user experience is fun. Either waiting for your box to startup, or when trying to win more NO2 by completing some goals, there is aways some cool interaction while you're coding. I've shared the join link with a lot of friends, both because they must try out the tool since it is a good one, and because they'll help me boost up my workspace ;)</div>
<h2>
And more</h2>
<div>
This is just a small overview, but you can also setup and connect using SSH, direct intor your box, allowing you to use a Local Eclipse when you need or you're tired to use the web IDE. You can also use your favorite DVCS tools, Mercurial and Git comes pre-installed by default. And a lot of more. See this page for a complete list of features: <a href="http://help.nitrous.io/box-interpreters-and-tools/">http://help.nitrous.io/box-interpreters-and-tools/</a></div>
<h2>
Conclusion</h2>
<div>
The <a href="https://www.nitrous.io/join/2Mbr_lb9O3M" rel="" target="_blank">Nitrous.IO</a> environment is fun, usefull, clean and solid. All you need to build your apps! Altought the princing is still not finished, and they're still in beta, Nitrous is the tool I'll definitelly keep looking forward, and I invite you to try it out.</div>
<div>
<br /></div>
<div>
Leave your comments bellow, and don't forget to subscribe to keep updated with the new posts of this series.</div>
Ronoaldo Pereirahttp://www.blogger.com/profile/02620222149209242748noreply@blogger.com0