Mobile Cloud Computing :::: Ecosystems & Research

Choosing Right AWS Database

2021-06-30T04:00:00.004-07:00

Choosing right Amazon Web Service (AWS) database is outstretched topic during analysis phase and beforehand of solution implementation of any AWS cloud-database based project. There are diverse parameters originate as strong-willed approach before finalizing the designed solution approach towards system architecture path on solving the problem statement. IMO, five components are being most matter to measure and build the right AWS database based solution far and wide, such as, operations, security, reliability, performance, and cost. In this article, I am simply comparing all AWS database services using above indicated components as baseline.

Most common questions to choose the right database based on your architecture:

1.        Read-heavy, write-heavy, or balanced workload? Throughput needs? Will it change, does it needs to scale or fluctuate during the day?
2.        How much data to store and for how long? Will it grow? Average object size? How are they accessed?
3.        Data durability? Source of truth for the data?
4.        Latency requirements? Concurrent users?
5.        Data model? How will you query the data? Joins? Structured? Semi-structured?
6.        Strong schema? More flexibility? Reporting? Search? RDBMS/NoSQL?
7.        License costs? Switch to Cloud Native DB such as Aurora? [Tip: To estimate the cost for your architecture solution, refer AWS Pricing Calculator]

Database Types:

1.        RDBMS (=SQL/OLTP):RDS, Aurora – great for joins
2.        NoSQL database: DynamoDB (~JSON), ElastiCache (key / value pairs), Neptune (graphs) – no joins, no SQL
3.        Object Store: S3 (for big objects) / Glacier (for backups / archives)
4.        Data Warehouse (= SQL Analytics / BI) : Redshift (OLAP), Athena
5.        Search: ElasticSearch (JSON) – free text, unstructured searches
6.        Graphs: Neptune – displays relationships between data

Sl. No.

Types of Database

Overview

Point to remember for Solutions Architect

Use Case

1

RDS

Managed PostgreSQL / MySQL / MariaDB / Oracle / SQL Server
Must Provision an EC2 instance & EBS Volume type and size
Support for Read Replicas and Multi-AZ
Security through IAM, Security, Groups, KMS, SSL in transit
Backup / Snapshot / Point in time restore feature
Managed and scheduled maintenance
Monitoring through Cloudwatch

Operations: small downtime when failover happens, when maintenance happens, scaling in read replicas / ec2 instance / restore EBS implies manual intervention, application changes
Security: AWS responsible for OS security, we are responsible for setting up KMS, security groups, IAM policies, authoring users in DB, using SSL
Reliability: Multi AZ feature, failover in case of failures
Performance: depends on EC2 instance type, EBS volume type, ability to add Read Replicas. Doesn’t auto-scale
Cost: Pay per hour based on provisional EC2 and EBS.

Store relational datasets (RDBMS/OLTP), perform SQL queries, transactional inserts/update/delete is available.

2

Aurora

Compatible API for PostgreSQL / MySQL
Data is held in 6 replicas, across 3 AZ – lot of durability
Auto healing capability
Multi AZ, Auto scaling Read Replicas
Read Replicas can be Global
Aurora database can be Global for DR or latency purposes
Auto scaling of storage from 10GB to 64TB
Define EC2 instance type for aurora instances
Same security / monitoring / maintenance features as RDS
“Aurora Serverless” option

Operations: less operations, auto scaling storage
Security: AWS responsible for OS security, we are responsible for setting up KMS, security groups, IAM policies, authoring users in DB using SSL
Reliability: multi AZ, highly available, possibly more than RDS, Aurora Serverless option
Performance: 5x performance (according to AWS) due to architectural optimization. Up to 15 Read Replicas (only 5 for RDS)
Cost: Pay per hour based on EC2 and storage usage. Possibly lower costs compared to Enterprise grade databases such as Oracle

Same as RDS, but with less maintenance / more flexibility / more performance.

*For enterprise grade applications, Aurora is the best option to choose.

3

ElastiCache

Managed Redis / Memcached (similar offering as RDS, but for caches)
In-memory data store, sub-millisecond latency
Must proviso an EC2 instance type
Support for Clustering (Redis) and Multi AZ, Read Replicas (sharding)
Security through IAM, Security groups, KMS, Redis Auth
Backup / Snapshot / Point in time restore feature
Managed scheduled maintenance
Monitoring through CloudWatch

Operations: same as RDS
Security: AWS responsible for OS security, we are responsible for setting up KMS, security groups, IAM policies, users (Redis Auth), using SSL
Reliability: multi AZ, Clustering, Sharding
Performance: Sub-millisecond performance, in memory, read replicas for sharding, very popular cache option
Cost: Pay per hour based on EC2 and storage usage.

Key/Value store, Frequent reads, less writes, cache results for DB queries, store session data for websites, cannot use SQL.

4

DynamoDB

DynamoDB is a pure cloud native technology, it’s a serverless
AWS proprietary technology, managed NoSQL database
Serverless, provisioned capacity, auto scaling, on demand capacity (Nov 2018) – scales based on your load
Can replace ElastiCache as a key/value store (storing session data for example)
Highly available, multi AZ by default, Reads and Writes are decoupled, DAX for read cache
Reads can be eventually consistent or strongly consistent
Security, authentication and authorization is done through IAM
DynamoDB streams to integrate with AWS Lambda
Backup / Restore feature, Global Table feature
Monitoring through CloudWatch
Can only query on primary key, sort key, or indexes

Operations: no operations needed, auto scaling capability, serverless
Security: full security through IAM policies, KMS encryption, SSL in flight
Reliability: multi AZ, Backups
Performance: single digit millisecond performance, DAX for caching reads, performance doesn’t degrade if your application scales
Cost: Pay per provisioned capacity and storage usage (no need to guess in advance any capacity – can use auto scaling).

Serverless applications development (small documents 100s KB), distributed serverless cache, doesn’t have SQL query language available, has transactions capability from NOV 2018.

5

S3

S3 is a database, it’s not a conventional database

S3 is a …key / value store for objects
Great for big objects, not so great for small objects
S3 doesn’t replace RDS / DynamoDB
Serverless, scales infinitely, max object size is 5TB
Eventually consistency for overwrites and deletes
Tiers: S3 Standard, S3 IA (Infrequent Access), S3 One Zone IA, Glacier for backups
Features: versioning, encryption, CRR (Cross Region Replication), etc…
Security: IAM, Bucket policies, ACL (Access Control List)
Encryption: SSE-S3, SSE-KMS, SSE-C, client side encryption, SSL in transit

Operations: no operations needed
Security: IAM, Bucket Policies, ACL, Encryption (Server/Client), SSL
Reliability: 99.999999999% durability / 99.99% availability, multi AZ, CRR
Performance: scales to thousands of reads / writes per second, transfer acceleration / multi-part for big files
Cost: pay per storage usage, network cost, request number

Static files, key value store for big files, website hosting.

6

Athena

It is not a database it terms but it holds the data but it does provide a query engine on top of S3.

Fully serverless database with SQL capabilities
Used to query data in S3
Pay per query
Output results back to S3
Secured through IAM

Operations: no operations needed, serverless
Security: IAM + S3 security
Reliability: managed service, used Presto* engine, highly available
Performance: queries scale based on data size
Cost: pay per query / per TB of data scanned, serverless

One time SQL queries, serverless queries on S3, log analytics.

7

Redshift

Redshift is based on PostgreSQL, but it’s not used for OLTP
It’s OLAP – online analytical processing (analytics and data warehousing)
10x better performance than other data warehouses, scale to PBs of data
Columnar storage of data (instead of row based)
Massively Parallel Query Execution (MPP), highly available
Pay as you go based on the instance provisioned
Has a SQL interface for performing the queries
BI tools such as AWS Quicksight or Tableau integrate with it
Data is loaded from S3, DynamoDB, DMS, other DBs…
From 1 node to 128 nodes, up to 160 GB of space per node
Leader node: for query planning, results aggregation
Compute node: for performing the queries, send results to leader
Redshift Spectrum: perform queries directly against S3 (no need to load)
Backup & Restore, Security VPC / IAM / KMS, Monitoring
Redshift Enhanced VPC Routing: COPY / UNLOAD goes through VPC

Operations: similar to RDS
Security: IAM, VPC, KMS, SSL (Similar to RDS)
Reliability: highly available, auto healing features
Performance: 10x performance vs other data warehousing, compression
Cost: pay per node provisioned, 1/10^th of the cost vs other warehouses

Remember: Redshift = Analytics / BI/ Data Warehouse

8

Neptune

Fully managed graph database
When do we use Graphs?
High relationship data
Social Networking: Users friends with Users, replied to comment on post of user and likes other comments
Knowledge graphs (Wikipedia)
Highly available across 3 AZ, with up to 15 read replicas
Point-in-time recovery, continuous backup to Amazon S3
Support for KMS encryption at rest + HTTPS

Operations: similar to RDS
Security: IAM, VPC, KMS, SSL (similar to RDS) + IAM Authentication
Reliability: Multi-AZ, clustering
Performance: best suited for graphs, clustering to improve performance
Cost: pay per node provisioned (similar to RDS)

Remember: Neptune = Graphs

9

ElasticSearch

With ElasticSearch, you can search any field, even partially matches
It’s common to use ElasticSearch as a complement to another database
ElasticSearch also has some usage for Big Data applications
You can provision a cluster of instances
Built-in integrations: Amazon Kinesis Data Firehose, AWS IoT, and Amazon CloudWatch Logs for data ingestion
Security through Cognito & IAM, KMS encryption, SSL & VPC
Comes with Kibana (visualization) & Logstash (log ingestion) – ELK stack

Operations: similar to RDS
Security: Cognito, IAM, VPC, KMS, SSL
Reliability: Multi-AZ, clustering
Performance: best on ElasticSearch project (open source), petabyte scale
Cost: pay per node provisioned (similar to RDS)

Remember: ElasticSearch = Search / Indexing

* Presto is a high performance, distributed SQL query engine for big data. It's architecture allows users to query a variety of data sources such as Hadoop, AWS S3, Alluxio, MySQL, Cassandra, Kafka, and MongoDB. One can even query data from multiple data sources within a single query.

CRUD-Operation-Handling: On Alibaba Cloud (Using ECS and RDS instance) over CentOS

2018-05-29T01:07:00.000-07:00

Hi Reader, in this tutorial, we'll learn how to deploy web application sample to Alibaba cloud. Please be noted the minimum deployment are involve here of using one ECS and one RDS instance.

To deploy web application on Alibaba cloud. I preferred to write a code in PHP language and built the CRUD operation where front-end s/w tools are installed on ECS and backend database deployed on RDS.

Following are the referral useful links to deploy the web application and to test it further as a pre-requisite:

Over the ECS instance – to run the front-end part of the web application sample, I've installed/configured following software and tools over the CentOS system image:

· Free Alibaba cloud account - https://www.alibabacloud.com/campaign/free-trial

· Apache: http://httpd.apache.org/

· PHP: http://www.php.net/

· MySQL: http://www.mysql.com/

· CentOS: http://www.centos.org/

· phpMyAdmin: http://www.phpmyadmin.net/

· Sample Web-application (CRUD based): https://github.com/RANBIJAY/CRUD-Operation-Alibaba-ECS-RDS

Suggestion: For best practice, it’s always recommended to execute and run the sample application at your own environment first (i.e., local machine), before deploying it into the target cloud services.

Before starting of our hands-on exercise, let’s do first benchmark set-up for both ECS and RDS services. During set-up process, users have a choice to opt either ‘subscription’ or ‘pay-as-go’ model depending on their needs.

I purchased super-high benchmark configuration for deployment of the sample web application and doing further R&D exercise in future assignment. The details are available below for reference:

1. Benchmarking of ECS

For ECS instance, from the offered list of System, Marketplace and Custom Image, I preferred to go with CentOS system image for hands-on, as I am more comfortable to Linux based environment rather than Windows. Also, I have Mac machine for doing my personnel R&D or innovation for my personnel interest.

Product

Elastic Compute Service

Instance Name:

i-a2daukxawmdw3athp40j

Configuration Details

Instance： 2-core, 8GB Generation III Network Enhanced sn2ne

IO optimized instance： IO optimized instance

System disk： /dev/xvda Ultra Cloud Disk

Bandwidth： 5.0000Mbps Data Transfer

CPU： 2Core

Available zone： Asia Pacific SOU 1 Zone A

OS： CentOS 7.3 64bit

Mem： 8GB

Region： Asia Pacific SOU 1 (Mumbai)

Network type： VPC

Manage service： Yes

After ECS instance get activated, the status of instance been turned “Running” in status.

To test it further, how it works? Just clicked on “Connect” option at the right side of the Instance list portal, it opens the “Management console” for doing further exercise. Before getting the screen you need to provide VNC connection password. (Alert: The VNC connection password is viewable only once. Ensure you record this password and keep it in a secure location for subsequent login attempts.)

2. Benchmarking of RDS

Product

ApsaraDB for RDS

Instance Name:

rm-6gjgil677k1r6xs1r

Configuration Details

RDS Specifications： 1 Core, 1 GB

Database type： MySQL

Database version： 5.7

Internet traffic： cost debited based on daily traffic usuage

Zone： Asia Pacific SOU 1 Zone A

Node type： High-Availability Edition

Region： Asia Pacific SOU 1 (Mumbai)

Storage： 5GB

Network type： VPC

My current ApsaraDB for RDS Management portal looks like below image. Before doing any exercise to make sure the status of instance should be “Running”.

Since, I opted MySQL DB to use for my hands-on exercise. Related to MySQL, quick start, set-up and configuration details are well documented and available in Alibaba Cloud Document center for reference, helped me to do my hands-on in an easy manner.

Set-up MySQL root database user account as below [Alert: Creating master account activation will disappear account/database management from RDS instance portal, once the master account gets activated. Best practice here to create first database and account (ordinary one) and give privilege Read/Write to access them]

Whitelist setting:

Other product also explored and used for hands-on purpose, such as:

· Object Storage Service (to upload/download the sample web app)

· Cloud-Monitor Service

Steps of Hands-on Exercise:

Step1: Installing Apache, PHP 7.2 and MySQL on CentOS 7.4 System ECS Instance

1. Note

In this exercise, I use the hostname/Public_IP address with the IP 149.129.134.33.

I will add the EPEL repo here to install latest phpMyAdmin as follows:

[root@ran-alibaba /]# rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY*

[root@ran-alibaba /]# yum -y install epel-release

To edit files on the shell, I'll install the nano editor. If you prefer vi/vim for file editing, then skip this step.

[root@ran-alibaba /]# yum -y install nano

2. Installing MySQL / MariaDB

MariaDB is a MySQL fork of the original MySQL developer Monty Widenius. MariaDB is compatible with MySQL and I've chosen to use MariaDB here instead of MySQL. Run this command to install MariaDB with yum:

[root@ran-alibaba /]# yum -y install mariadb-server mariadb

Then we create the system startup links for MySQL (so that MySQL starts automatically whenever the system boots) and start the MySQL server:

[root@ran-alibaba /]# systemctl start mariadb.service

[root@ran-alibaba /]# systemctl enable mariadb.service

Set passwords for the MySQL root account:

[root@ran-alibaba /]# mysql_secure_installation

In order to log into MariaDB to secure it, we'll need the current password for the root user. If you've just installed MariaDB, and you haven't set the root password yet, the password will be blank, so you should just press enter here.

Enter current password for root (enter for none): <--ENTER

Ok, successfully used password, moving on...

Setting the root password ensures that nobody can log into the MariaDB root user without the proper authorization.

[root@ran-alibaba /]# Set root password? [Y/n]

New password: <--yourmariadbpassword

Re-enter new password: <--yourmariadbpassword

Password updated successfully!

Reloading privilege tables.

... Success!

By default, a MariaDB installation has an anonymous user, allowing anyone to log into MariaDB without having to have a user account created for him or her. This is intended only for testing, and to make the installation go a bit smoother. You should remove them before moving into a production environment.

Remove anonymous users? [Y/n] <--ENTER

... Success!

Normally, root should only be allowed to connect from 'localhost'. This ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? [Y/n] <--ENTER

... Success!

By default, MariaDB comes with a database named 'test' that anyone can access. This is also intended only for testing, and should be removed before moving into a production environment.

Remove test database and access to it? [Y/n] <--ENTER

- Dropping test database...

... Success!

- Removing privileges on test database...

... Success!

Reloading the privilege tables will ensure that all changes made so far will take effect immediately.

Reload privilege tables now? [Y/n] <--ENTER

... Success!

Cleaning up...

All done! If you've completed all of the above steps, your MariaDB installation should now be secure.

Thanks for using MariaDB!

[root@ran-alibaba /]#

3. Installing Apache

CentOS 7 ships with Apache 2.4. Apache is directly available as a CentOS 7 package, therefore we can install it like this:

[root@ran-alibaba /]# yum -y install httpd

Now configure your system to start Apache at boot time...

[root@ran-alibaba /]# systemctl start httpd.service

[root@ran-alibaba /]# systemctl enable httpd.service

To be able to access the web server from outside, we have to open the HTTP (80) and HTTPS (443) ports in the firewall. The default firewall on CentOS is firewalld, which can be configured with the firewalld-cmd command.

[root@ran-alibaba /]# firewall-cmd --permanent --zone=public --add-service=http

[root@ran-alibaba /]# firewall-cmd --permanent --zone=public --add-service=https

[root@ran-alibaba /]# firewall-cmd --reload

Now direct your browser to the IP address of your server, in my case http://149.129.134.33/, and you should see the Apache placeholder page:

4. Installing PHP

The PHP version that ships with CentOS as default is quite old (PHP 5.4). Therefore I will show you in this exercise some options to install newer PHP versions like PHP 7.0 or 7.1 from Remi repository.

Add the Remi CentOS repository.

[root@ran-alibaba /]# rpm -Uvh http://rpms.remirepo.net/enterprise/remi-release-7.rpm

[root@ran-alibaba /]# Install yum-utils as we need the yum-config-manager utility.

[root@ran-alibaba /]# yum -y install yum-utils

and run yum update

[root@ran-alibaba /]# yum update

Now you have an option to choose which PHP version you want to use on the server. PHP version at a time with Apache mod_php. To install PHP, run this command:

[root@ran-alibaba /]# yum -y install php

To Install PHP 7.2, use:

[root@ran-alibaba /]# yum-config-manager --enable remi-php72

[root@ran-alibaba /]# yum -y install php php-opcache

We must restart Apache to apply the changes:

[root@ran-alibaba /]# systemctl restart httpd.service

5. Testing PHP / Getting Details About Your PHP Installation

The document root of the default website is /var/www/html. We will create a small PHP file (info.php) in that directory and call it in a browser to test the PHP installation. The file will display lots of useful details about our PHP installation, such as the installed PHP version.

[root@ran-alibaba /]# nano /var/www/html/info.php [install nano editor on CentOS: yum -y install nano]

In nano editor, type following code and exit.

<?php

phpinfo();

To switch from nano editor to root-command, in Mac OSX: press fn+F2. Now we call that file in a browser (e.g. http://149.129.134.33/info.php):

As you see, PHP 7.2 is working, and it's working through the Apache 2.0 Handler, as shown in the Server API line. If you scroll further down, you will see all modules that are already enabled in PHP. MySQL is not listed there which means we don't have MySQL support in PHP yet.

6. Getting MySQL Support In PHP

To get MySQL support in PHP, we can install the php-mysqlnd package. It's a good idea to install some other PHP modules as well as you might need them for your applications. You can search for available PHP5 modules like this:

[root@ran-alibaba /]# yum search php

Pick the ones you need and install them like this:

[root@ran-alibaba /]# yum -y install php-mysqlnd php-pdo

In the next step I have installed some common PHP modules that are required by CMS Systems like Wordpress, Joomla, and Drupal (for future exercise and learning):

[root@ran-alibaba /]# yum -y install php-gd php-ldap php-odbc php-pear php-xml php-xmlrpc php-mbstring php-soap curl curl-devel

Now restart Apache web server:

[root@ran-alibaba /]# systemctl restart httpd.service

Now reload http://149.129.134.33/info.php in your browser and scroll down to the modules section again. You should now find lots of new modules like curl etc. there:

If you don't need the php info output anymore, then delete that file for security reasons.

[root@ran-alibaba /]# rm /var/www/html/info.php

7. phpMyAdmin installation

phpMyAdmin is a web interface through which you can manage your MySQL databases.

phpMyAdmin can now be installed as follows:

[root@ran-alibaba /]# yum -y install phpMyAdmin

Now we configure phpMyAdmin. We change the Apache configuration so that phpMyAdmin allows connections not just from localhost (by commenting out the <RequireAny> stanza and adding the 'Require all granted' line):

[root@ran-alibaba /]# nano /etc/httpd/conf.d/phpMyAdmin.conf

Inside the nano editor, the screens should be

[...]

Alias /phpMyAdmin /usr/share/phpMyAdmin

Alias /phpmyadmin /usr/share/phpMyAdmin

<Directory /usr/share/phpMyAdmin/>

AddDefaultCharset UTF-8

<IfModule mod_authz_core.c>

# Apache 2.4

# <RequireAny>

# Require ip 127.0.0.1

# Require ip ::1

# </RequireAny>

Require all granted

</IfModule>

<IfModule !mod_authz_core.c>

# Apache 2.2

Order Deny,Allow

Deny from All

Allow from 127.0.0.1

Allow from ::1

</IfModule>

</Directory>

<Directory /usr/share/phpMyAdmin/>

Options none

AllowOverride Limit

Require all granted

</Directory>

[...]

Next, we change the authentication in phpMyAdmin from cookie to http:

[root@ran-alibaba /]# nano /etc/phpMyAdmin/config.inc.php

[...]

$cfg['Servers'][$i]['auth_type'] = 'http'; // Authentication method (config, http or cookie based)?

[...]

Restart Apache:

[root@ran-alibaba /]# systemctl restart httpd.service

Afterwards, you can access phpMyAdmin underhttp://149.129.134.33/phpmyadmin/:

Step2: phpmyadmin_connect_to_rds [Tutorial on connecting your phpmyadmin to an external Cloud RDS instance]

[root@ran-alibaba /]# nano /etc/phpMyAdmin/config.inc.php

--ADD LINES BELOW THE PMA CONFIG AREA AND FILL IN DETAILS--

$i++;

$cfg['Servers'][$i]['host'] = '<RDS-Endpoint>';

$cfg['Servers'][$i]['port'] = '3306';

$cfg['Servers'][$i]['socket'] = '';

$cfg['Servers'][$i]['connect_type'] = 'tcp';

$cfg['Servers'][$i]['extension'] = 'mysql';

$cfg['Servers'][$i]['compress'] = FALSE;

$cfg['Servers'][$i]['auth_type'] = 'config';

$cfg['Servers'][$i]['user'] = '<Username>';

$cfg['Servers'][$i]['password'] = '<Password>';

Step3: Testing Web-Application URL:

http://149.129.134.33/ran-pdo-master/public/

Screen-shot UI:

Backend:

Building Alexa Skills (or apps) through Swift programming language is possible?

2017-04-09T06:49:00.001-07:00

Yes, absolutely it’s possible. As we few of many already know that Swift-programming language allow iOS developers to build or develop iOS based applications. Here out of the box, we’ll experience the way of methodology to expand Swift proficiency skill set to include server-side programming and take part in the trend towards voice user interfaces (i.e., Amazon Echo). Please be noted, currently AWS Lambda (i.e., server-less compute) only supports code written in JavaScript (Node.js), Python, C# and Java. But it’s easy to extend this to executable written in any programming language you want.

In this post we will learn and understand about methodology how to build or develop Alexa skills (or app) with the proficiency of ‘Swift’ programming language? Also achieve an idea of integrating created skills with other AWS cloud services too. Before jump into the context, I really recommend here to check it out or follow below highlighted links – will help to explore and understand the basic implementation guidelines. Reading or follow amazon documentation will be plus to achieve the target implementation result.

Important Links:

· https://github.com/IBM-Swift/Kitura

· https://github.com/IBM-Swift/HeliumLogger

· https://github.com/choefele/AlexaSkillsKit

· https://docs.docker.com/docker-for-mac/ - Getting started with Docker for Mac

Blog Link:

· Alexa skills from scratch with AWS Lambda, SSML and Amazon S3

· Swift Programming (Server-Side) - Kick Start..!

Sample tested on below system-setup:

· macOS Sierra – 10.12.4

· XCode – 8.3 (8E162)

· Swift – 3.1

· Docker – Community Edition - Version 17.03.0-ce-mac2 (15654)

Objective Target:

Alexa will send created skill a JSON message with the user’s intent and sample code answers with a JSON message that determines what Alexa will answer to the user.

The code for our custom skill can run as either a stand-alone web service (i.e., HTTP) or an AWS Lambda function. Using Lambda, Amazon’s server-less computing platform, Amazon will take care of scaling and running sample Swift code.

1. HTTPS

Stand-alone web service (i.e., HTTP)

$ cd /Users/ranbijaykumar/Alexa-AVS-Sample/Alexa-Swift-Skills/swift-lambda-app - using {Reference-Implementation} below instead of the mentioned directory path.

2. AWS Lambda ARN (Amazon Resource Name)

AWS Lambda is a server-less compute service that runs code in response to events and automatically manages the underlying compute resources.

Importantly I would like to mention here that I pulled the source sample code from Github – developed by Claus Höfele. Though the sample code is straightforward but I found personally few gap during time of implementation – bridged and tried my best to make it easier from developer perspective.

Either clone or download the sample app from Github account. Else, check out the swift-lambda-app repo on GitHub for the code and scripts to develop and deploy a simple Alexa skill in Swift via Terminal command prompt in MacOS.

· $ git clone https://github.com/choefele/swift-lambda-app.git

· $ swift build

· $ swift package generate-xcodeproj

· $ open ./AlexaSkill.xcodeproj/

We’ll need to understand the following for our Alexa skill:

· An implementation of skill’s functionality in Swift using AlexaSkillsKit

· A Lambda function set up with your Swift code using the AWS Console

· An Alexa Skill configured in the Alexa Console that triggers your Lambda function

*Note that the Alexa Console and the AWS Console are two separate services that user need to sign up for.

There are three targets:

· AlexaSkill: this is a library with the code that implements the custom Alexa skill. It’s a separate library so the other two targets can use it. Also, libraries have ENABLE_TESTABILITY enabled by default, which allows user to access internal methods and properties in your unit tests.

· Lambda: The command line executable for deployment to Lambda. This program uses stdin and stdout for processing data.

· Server (macOS only): To simplify implementing a custom Alexa Skill, the server provides an HTTP interface to the AlexaSkill target. This HTTP server can be exposed publicly via ngrok and configured in the Alexa console, which enables you to develop and debug an Alexa skill with code running on our development computer. This target is macOS only.

HTTPS: To run a local HTTPS server

· Make sure the sample builds by running swift build

· Generate an Xcode project with swift package generate-xcodeproj

· Open the generated Xcode project, select the Server scheme and run the product (CMD-R). This will start a server at port 8090

· Install ngrok via brew cask install ngrok. This tool allows you to expose a local HTTP server to the internet

§ $ cd {Reference-Implementation}

§ $ brew cask install ngrok -à Install ngrok

· Run ngrok http 8090 and copy the HTTPS URL generated by ngrok (it looks similar to https://6ebec3f2.ngrok.io)

§ $ ngrok http 8090 -à ‘Ctrl+C’ to get back to command prompt.

* ngrok exposes your local server to the public internet thus allowing the Alexa Voice Service to call into your custom skill running in Xcode.

Configuring the Alexa Skill

To hook up your custom skill to Alexa:

· Go to the Alexa console and create a new skill

· Skill type: Custom Interaction Model

· Intent:

{"intents":
[{"intent": "TestIntent"}]}

· Sample utterances: “TestIntent test Swift”

· Service endpoint type: HTTPS (use the URL from ngrok) - Copy above created HTTPS URL (i.e., in format of https://6ebec3f2.ngrok.io) and paste into the below endpoint field as depicted below. Press ‘Next’

· SSL Certificate: Select “My development endpoint is a sub-domain of a domain that has a wildcard certificate from a certificate authority”

Now we can test the skill in the Alexa Console’s Service Simulator using the utterance “test swift”. This will call your local server allowing you to modify and debug your code while interacting with the Alexa service. You can check the status ‘200’ into local running HTTPS server over ngrok terminal. Please check above in diagram.

Running Tests for Your Code

Before uploading to Lambda, it’s worthwhile to run your unit tests in a Linux environment and run integration tests that simulate the execution environment. The sample provides run-unit-tests.sh to do the former and run-integration-tests.sh to do the latter.

run-unit-tests.sh builds and tests the Lambda target inside a Swift Docker container based on Ubuntu because there's currently no Swift compiler for Amazon Linux (based on RHEL). Executables built on different Linux distributions are compatible with each other if you provide all dependencies necessary to run the program. For this reason, the script captures all shared libraries required to run the executable using ldd.

To prove that the resulting package works, run-integration-tests.sh runs a release build of the Swift code inside a Docker container that comes close to Lambda’s execution environment (unfortunately, Amazon only provides a few Docker images that don't necessarily match what Lambda is using).

The integration with Lambda is done via a small Node.js script that uses the child_process module to run the Swift executable. The script follows Amazon's recommendations to run arbitrary executables in AWS Lambda.

*After configuration, we need to run the same integration script again for every commit or possible changes.

Deploying Sample Code to Lambda

To deploy sample code to AWS Lambda:

· Before jumping to below steps, first run the Docker container, Install Docker S/W in development machine from above link and check the status of Docker container – as depicted above screenshot.

· Run run-integration-tests.sh to produce a zip file at .build/lambda/lambda.zip with all required files to upload to Lambda

$ cd {Reference-Implementation}

$ mkdir -p .build/lambda/libraries/

$ ./run-unit-tests.sh

$ ./run-integration-tests.sh

It will create the .zip in hidden folder - Hold down CMD-Shift-. (dot) keys in Mac machine --à All the hidden files will become visible. Press same group of keys again to make the hidden files/folder disappear as previous.

· Create a new Lambda function in the AWS Console in the US East/N. Virginia region (for Europe use EU/Ireland). Select ‘alexa-skill-kit-sdk-factskill’ blueprint

· Use an Alexa Skills Kit trigger

· Runtime: NodeJS 4.3 or NodeJS 6.10

· Code entry type: ZIP file (upload the lambda.zip file from the previous step)

· Handler: index.handler

· Role: Create from template or use existing role ‘lambda_basic_execution’

Once we uploaded the Lambda function, we can use the test actions in the AWS Console, for example by using a Start Session action.

Configuring the Alexa Skill for Lambda

After creating the Lambda function, you can now create an Alexa skill. If you have previously created an Alexa skill for the local HTTP server — the only difference is the service endpoint:

· Go to the Alexa console and create a new skill

· Skill type: Custom Interaction Model

· Intent:

{"intents":
[{"intent": "TestIntent"}]}

· Sample utterances: “TestIntent test swift”

· Service endpoint type: AWS Lambda ARN (use the ARN for the Lambda function from the AWS Console) – ARN: arn:aws:lambda:us-east-1:280019173811:function:swiftDeploymentInLambda. See previous of previous image to specified/copied the created ARN value at endpoint field. [Please refer above mentioned blog links as well to understand more into the deep].

Now we can test the skill in the Alexa Console using the utterance “test swift”. More details on configuring Alexa skills can be found on Amazon’s developer portal.

Done J

I hope this post will give you a great kick-of start using Swift programming language in both client and server side on creating of an Alexa skills, also integrating those skills with other AWS cloud services seamlessly. Thanks you and good luck, happy programming!

Amazon Alexa Voice Service (AVS): Run and Test Sample app on Mac – Java Client and iOS Simulator

2017-03-19T09:00:00.001-07:00

Hi Reader,

Earlier I talked about how to create and develop Alexa Skills from scratch and to use it around AWS Lambda function, SSML and Amazon S3 storage. Also, I recommended using echoism.io cloud service for testing the Alexa skills. Here you can refer to learn and start with. Today, I am delighted to share the very straightforward practical approach of using Alexa Skills and Alexa Voice Service (AVS) together – even we could find better accuracy and performance with that especially into the client section side. For this post, I restricted or tested app over Java Client and iOS simulator into Mac machine. More importantly, with this post, we can learn to build or enabled the Alexa skills without dependency of real Amazon Echo hardware (i.e., personal assistant). Let’s start:

Important Links to refer for implementation the AVS (Bring your capabilities to connected devices) skill app:

· https://github.com/alexa

· https://github.com/alexa/alexa-avs-sample-app/wiki/Mac

· https://github.com/alexa/alexa-avs-sample-app/wiki/Create-Security-Profile

With this post, I’ll highlight the gaping experience takeaway points – should be nice to include into the content of above links for developer comfort and understanding. Though the mentioned links are straightforward and have complete steps for implementation, running or testing the AVS sample app, but at the certain point being a developer I found little difficulty with that:

We need to first follow environment set-up checklist into our development machine before jumping into the coding section:

Install New Software

Ensure you have JDK version 8 installed. If JDK is not installed or you need to update from a previous version, you can download version 8 here. Note: As mentioned earlier, please ensure that your JDK's architecture matches the architecture for VLC. The sample worked only in JDK 1.8.
Download VLC media player here.
Download and install Node.js.
Download Maven and follow the instructions for Installing Apache Maven.
If you plan to authenticate using an iOS companion app, download Xcode for Mac from the Mac App Store.
Also install or update Homebrew (brew) package manager to install OpenSSL (openssl), NVM, Maven (mvn) package via Terminal.

Once finished with the installations – check version of latest tools/software into the Mac-Terminal and update all if it’s already been installed earlier in system.

After you check the version of installed tools/software, set the following into the environment variables via your favourite editor:

$ open ~/.bash_profile

Once you set the path, save the file and close it. Come back to the terminal:

$ source ~/.bash_profile

Getting Started with the Alexa Voice Service (AVS)

Download the sample app into the project directory

$ mkdir /Users/ranbijaykumar/Alexa-AVS-Sample

Note: /Users/ranbijaykumar/Alexa-AVS-Sample/. Further instructions will reference this location as <REFERENCE_IMPLEMENTATION>

$ cd /Users/ranbijaykumar/Alexa-AVS-Sample

$ git clone https://github.com/alexa/alexa-avs-sample-app.git (Hope you have already installed github package)

*By downloading this package, you agree to the Alexa Voice Service Agreement.

Register for a free Amazon Developer Account:

Get a free Amazon developer account if you do not already have one: https://developer.amazon.com/login.html

Register your product and create a security profile.

1. Login to Amazon Developer Portal - developer.amazon.com

2. Click on Apps & Services tab -> Alexa -> Alexa Voice Service -> Get Started

3. In the Register a Product Type menu, select Device

4. Fill in and save the following values:

Device Type Info:

· Device Type ID: my_device  

· Display Name: My Device  Then click Next  

Security Profile

5. Click on the Security Profile dropdown and choose “Create a new profile”:  

6. General Tab

· Security Profile Name: Alexa Voice Service Sample App Security Profile  

· Security Profile Description: Alexa Voice Service Sample App Security Profile Description  

· Click Next  

The Client ID and Client Secret will be generated for you.  

7. Now click on the Web Settings Tab

· Make sure the security profile you just created is selected in the drop-down menu, then click   the "Edit" button.  

· Allowed Origins: Click "Add Another" and then enter https://localhost:3000 in the text field that appears.  

· Allowed Return URLs: Click "Add Another" and then enter https://localhost:3000/authresponse in the text field that appears.  

· Click Next  

8. Device Details

[Optional] Image: Pick the test image to your computer, then upload it:  

Category: Other  

Description: Alexa Voice Service sample app test  

Do you have plans to make your product available to the general public? - No

Click Next

9. We are now ready to generate self-signed certificates

Open a web browser, and visit https://developer.amazon.com/lwa/sp/overview.html.
Near the top of the page, select the security profile you created earlier from the drop down menu and click Confirm.
Enter a privacy policy URL beginning with http:// or https://. For this example, you can enter a fake URL such as http://example.com.
[Optional] You may upload an image as well. The image will be shown on the Login with Amazon consent page to give your users context.

5. Click Save.

Next to the Alexa Voice Service Sample App Security Profile, click Show Client ID and Client Secret. This will display your client ID and client secret. Save these values on Clipboard. We’ll need these.

*Also refer here for above steps with screenshot - Create a device and security profile

Generate self-signed certificates

Change directories to <REFERENCE_IMPLEMENTATION>/samples/javaclient. In this instance, the <REFERENCE_IMPLEMENTATION> is ”/Users/ranbijaykumar/Alexa-AVS-Sample/alexa-avs-sample-app/”.

Step 1: Edit the text file ssl.cnf, which is an SSL configuration file. Fill in appropriate values in place of the placeholder text that starts with YOUR_.

$cd <REFERENCE_IMPLEMENTATION>/samples/javaclient - //your sample apps location

$ edit ssl.cnf , will open file in your desired editor

Note: countryName must be two characters (e.g. US). If it is not two characters, certificate creation will fail. Additionally, if you will be accessing your device from any IP or DNS entry besides localhost (127.0.0.1 or 10.0.2.2), you must add the additional IP or or DNS entries to [alt_names]. One situation where you will need to add entries to [alt_names] is if you are going to authenticate using an Android or iOS companion app from a device instead of from the Android or iOS emulators on the same machine as the Node.js server and sample app. Here's what the ssl.cnf file would look like, replacing country, state, locality with your respective info.

Save the file.

Step 2: Make the certificate generation script executable by typing:

$ chmod +x generate.sh

Step 3: Run the certificate generation script:

$ ./generate.sh

Step 4: You will be prompted for some information:

· When prompted for a product ID, enter the productID (or Device Type ID) listed under "Device Type Info" in the Amazon developer portal. For this prototype enter my_device

· When prompted for a serial number, enter your product's serial number. For this prototype enter 123456  

· When prompted for a password, enter any password and remember what you entered   (you can even leave it blank). For this prototype leaving password field blank.

Step 5: Edit the configuration file for the Node.js server

The configuration file is located at: <REFERENCE_IMPLEMENTATION>/samples/companionService/config.js. Make the following changes:

$ edit config.js

Set clientID: paste in the client ID as a string that we noted in the earlier step while creating “security profile” from here.  

Set clientSecret: paste in the client ID as a string that we noted in the earlier step while creating “security profile” from here.

Set sslKey to <REFERENCE_IMPLEMENTATION>/samples/javaclient/certs/server/node.key  

Set sslCert to <REFERENCE_IMPLEMENTATION>/samples/javaclient/certs/server/node.crt  

Set sslCaCert to <REFERENCE_IMPLEMENTATION>/samples/javaclient/certs/ca/ca.crt

· Set products: The product's object consists of a key that should be the same as the product type ID (also referred as Device Type ID) that we set up in the developer portal and a value that is an array of unique product identifiers. If we followed the instructions above, the product type ID should be my_device. The unique product identifier can be any alphanumeric string, such as 123456. Example products JSON is: products: {"my_device": ["123456"]}. For this project, the array should be a single value, and match the serial number you entered while generating certificates. Please be noted the password field is blank.  

Here’s what config.js file look like:

 IMPORTANT: Do not use ~ to denote the home directory. Use the absolute path instead. So in this case, use ”/Users/ranbijaykumar/Alexa-AVS-Sample/alexa-avs-sample-app/…”Save the file.  

Step 6: Edit the configuration file for the Java client  The configuration file is located at: <REFERENCE_IMPLEMENTATION>/samples/javaclient/config.json.

$ edit config.json

Make the following changes:  

Set productId: Enter my_device as a string.  

Set dsn: Enter the alphanumeric string that you used for the unique product identifier in the products object  in the server's config.js. For example: 123456.  

Set provisioningMethod: If you want to use either the Android or iOS sample app, enter companionApp. Otherwise, enter companionService

Set companionApp.sslKeyStore to <REFERENCE_IMPLEMENTATION>/samples/javaclient/certs/server/jetty.pkcs12  

Set companionApp.sslKeyStorePassphrase to the passphrase entered in the certificate generation script in step 4 above. For this prototype password field is blank. So leave as it is.  

Set companionService.sslClientKeyStore to <REFERENCE_IMPLEMENTATION>/samples/javaclient/certs/client/client.pkcs12  

Set companionService.sslClientKeyStorePassphrase to the passphrase entered in the certificate generation script in step 4 above. For this prototype password field is blank. So leave as it is.

Set companionService.sslCaCert to <REFERENCE_IMPLEMENTATION>/samples/javaclient/certs/ca/ca.crt

Here’s what config.json file look like (this file created for iOS Companion i.e., provisioningMethod: ”companionApp”)

Choose Authentication Method

We should now decide if we want our client users to authenticate using a website or mobile app. We can only authenticate using one of the following methods at a given time:

· IMPORTANT: Only one method can be used at a time. If we want to test authentication using a web site and a mobile companion app (iOS or Android), we can do so by testing the first provisioning method, then clearing your access token and switching to the other provisioning method. For instructions see Clearing Your Access Token to Change Provisioning Methods.

1) Build and Run a Node.js Server Authentication

Install the Companion Service dependencies

Change directories to <REFERENCE_IMPLEMENTATION>/samples/companionService

$ cd <REFERENCE_IMPLEMENTATION>/samples/companionService
Install the dependencies by typing:

$ npm install

Run the following:

$ npm start

The server is running. You are now ready to run the sample app. The server is now running on port 3000 and you are ready to start the client.

2) Build and Run an iOS Companion App for Authentication:

To see an example of how to obtain a Login with Amazon authorization code from an iOS app, build and run the iOS sample project.

To build and run the iOS mobile app, follow these instructions:

Open the iOS sample app in Xcode. To do this, double click <REFERENCE_IMPLEMENTATION>/samples/iOSCompanionApp/AlexaCompanionAppSample.xcodeproj
Log in to the Amazon developer portal.
In the top navigation, click Alexa then select Alexa Voice Service. Find your project, and click Edit, then navigate to the Security Profile tab.
From the drop down, select the security profile you created in Getting Started.
Click the iOS Settings tab.
Enter the following information:

API Key Name: "Alexa Voice Service test iOS app"
Bundle ID: com.amazon.AlexaCompanionAppSample

Click Add an API Key
Copy the key to your computer's clipboard.
Add the API key to the Xcode project.

In the Xcode project, make sure the left navigation pane is set to the project navigator by clicking the left-most icon that looks like a file folder at the top of the navigation pane.
Expand AlexaCompanionAppSample → Application → Supporting Files, then select AlexaCompanionAppSample-Info.plist.
In the middle editing pane, make sure none of the entries are selected.
In the menu bar, select Editor → Add Item.
In the combo drop down that appears, type APIKey and press return.
Double click within the Value column and paste the API key that you copied in point 8. Make sure there is no extra whitespace before and after the value, then press return.

Build and run the app in the simulator by clicking the black right-pointing play button at the top left of the Xcode window.

We are now ready to run the sample app.

Run the Java-Client Sample App:

Depending on whether you specified companionApp or companionService as the provisioningMethod in the config.json, follow the instructions below for -

Using an Android or iOS App to Obtain Authorization from Login with Amazon, or
Using the Java Client to Obtain Authorization from Login with Amazon.

In text editor, open <REFERENCE_IMPLEMENTATION>/samples/javaclient/pom.xml and locate {alpn-boot.version}xxx{/alpn-boot.version}. Confirm that the ALPN version matches your JDK version using the table located here. If the versions match no further action is necessary. If the versions do not match, update the pom.xml file with the correct ALPN version.IMPORTANT: If your JDK version is newer than what's listed in the table, please use the latest ALPN version listed.

Open a terminal window.
Change directories to <REFERENCE_IMPLEMENTATION>/samples/javaclient
Before you build the app, let’s validate to make sure the project is correct and that all necessary information is available. You do that by running: Run mvn validate to ensure the project is correct and that all necessary information is available.

$ mvn validate

4. Download dependencies and build the app by typing: (you might want to close any unnecessary apps or you might run into an error due to insufficient memory) .Run mvn install to download dependencies and build the sample app.
$ mvn install

5. When the installation is completed, we will see a “Build Success” message in the terminal. Run mvn exec:exec to run the client sample app.

$ mvn exec:exec

Obtain Authorization from Login with Amazon

1. When you run the client, a window should pop up with a message that says something similar to:

Copy the URL from the popup window and paste it into a web browser. In this example, the URL to copy and paste is https://localhost:3000/provision/4d3723cf95197f806dfb4af0.

NOTE: Due to the use of a self-signed certificate, we will get a warning about an insecure website. This is expected. It is safe to ignore the warnings during testing.

2. We will be taken to a Login with Amazon web page. Enter Amazon credentials.

Please register your device by visiting the following website on any system and following the instructions: https://localhost:3000/provision/4d3723cf95197f806dfb4af0 Hit OK once completed.

3. We will be taken to a Dev Authorization page, confirming that we’d like your device to access the Security Profile created earlier. Click Okay.  

4. We will now be redirected to a URL beginning with https://localhost:3000/authresponse followed by a query string. The body of the web page will say device tokens ready.  

5. Return to the Java application and click the OK button. The client is now ready to accept Alexa requests.  

6. Wait a few seconds for the Token to appear in the Alexa Voice Service window. Click the Start Listening button and wait for the audio cue before beginning to speak. It may take a second or two for the connection to be made before you hear the audio cue.  

7. Press the Stop Listening button when we are done speaking.

Let’s talk to Alexa

Ask for Weather: Click the Start Listening button.

You: What's the weather in Seattle? Click the Stop Listening button.

Alexa: Current weather report for Seattle

Some other fun questions we can ask to Alexa

Once we hear the audio cue after clicking “Start Listening” button, here are a few things available we can try saying -

Request Music Playback: Play Bruce Springsteen  

General Knowledge: What's the mass of the sun in grams?  

Geek: What are the three laws of robotics?  

Fun: Can you rap?  

Set a Timer: Set the timer for 2 minutes.  

Set Alarm: Set the alarm for 7:30 a.m.  More on Music Playback The "previous", "play/pause", and "next" buttons at the bottom of the Java client UI are to demonstrate the music button events. Music button events allow you to initiate changes in the playback stream without having to speak to Alexa. For example, we can press the "play/pause" button to pause and restart a track of music.  To demonstrate the "play/pause" button, we can speak the following command: Play DC101 on iHeartRadio, then press the "play/pause" button. The music will pause in response to the button click. Press the "play/pause" button again to restart the music.  

*Even we can call our own created Alexa skills.

Run the iOS Companion App:

If our provisioning method is companionApp and you followed the instructions above to build and run the iOS app, follow these instructions:

The iOS app should be running in the iOS simulator on the same computer where we will run the sample app.
In the simulator, enter the following address for the device address, then click Connect:

iOS: https://127.0.0.1:8443

A gold Login with Amazon button should appear on the screen. Click the button.
Enter Amazon login information and click Sign in.
In the mobile app, we will see a message that says Your device is now successfully provisioned and ready to use. In the sample app, the Bearer Token text field should now be filled-in with access token. The sample app is now ready to accept requests.

Note: we should only attempt to log in to Login with Amazon from either the Android app or the iOS app, not both at the same time. If we want to test out each app separately, once we have successfully tested one of the apps, follow the instructions below for clearing your access token and then test the other app.

We're now ready to talk to Alexa via iOS Simulator.

Note: For handy purpose, uploaded entire contents of this post in PDF format over here.

Thanks for holding patience to learn a big or elaborated most blog content, hope you find it useful for your development usage. Cheers!

Stripe Payment Solution on iOS platform - Using Swift

2017-03-10T01:48:00.000-08:00

Hi Everyone,

In this blog we’ll see the integration methodology of Stripe payment API with iOS platform based application.

To start with, we required first to set-up an environment in development machine and look at some important key steps during time of the implementation. I have done this lab sample in my Mac machine i.e., the configuration spec are such as:

· Mac OSX- 10.12.3

· XCode – 8.2.1

· Swift – 3.0.2

· Getting start with the Stripe-Mobile: https://stripe.com/docs/mobile

1. First create a new project in Xcode:

Open Xcode IDE - > File -> New -> Project - > Choose ‘Single View Application’ template under iOS for creation of new project-> provide desired ‘Product Name’ and pick ‘Swift’ corresponding to ‘Language’ selection and ‘Devices’ should be ‘iPhone’ at this moment (can be considered ‘Universal’ as well for supporting iPad in addition).

E.g. – the project directory path look like: /Users/ranbijaykumar/Stripe-Sample/Stripe-iOS

2. Next, open Terminal in Mac machine:

Install the latest version of ‘Cocoapods’

$ sudo gem install cocoapods

Once you installed the ‘Cocoapods’ in machine. Go to the directory of your project and create podfile:

$ cd /Users/ranbijaykumar/Stripe-Sample/Stripe-iOS

$ init pod (please make sure the Podfile should create in same directory of the project where .xcodeproj and .xcworkspace are already there)

$ edit pod

Open Podfile, add this line to the Podfile i.e., pod ‘Stripe’. This will be look like it:

# Uncomment the next line to define a global platform for your project

# platform :ios, '9.0'

target 'Stripe-iOS' do

# Comment the next line if you're not using Swift and don't want to use dynamic frameworks

use_frameworks!

# Pods for Stripe-iOS

pod 'Stripe'

pod 'CardIO'

pod 'SVProgressHUD'

end

Save and close the Podfile. Run the following command under project directory folder in Terminal:

$ pod install (latest Stripe SDK (i.e., Stripe – 9.4.0 will be installed in your Xcode project)

* For future - to update the installed SDK version $ pod update, to do that - make sure first to remove pod.lock file from the project directory.

Run the following command in terminal under project directory in case you have closed your Xcode IDE.

$ open ./Stripe-iOS.xcworkspace

3. Open AppDelegate.swift file of your project in Xcode

import UIKit

import Stripe

@UIApplicationMain

class AppDelegate: UIResponder, UIApplicationDelegate {

var window: UIWindow?

func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplicationLaunchOptionsKey: Any]?) -> Bool {

// Override point for customization after application launch.

        STPPaymentConfiguration.shared().publishableKey = "pk_test_3m5KLiVZ2BsiRJVH3Xsx5ViW"

return true

}

To get a Test and Live ‘Secret and Publishable key’, user need to register over here: https://dashboard.stripe.com/register. The publishable key can be replaced in above statement or track the logs via dashboard for future transactions or any actions.

4. Under ViewController.swift: paste below code, allow users to fetch the default UIView for Stripe Payment option and getting tokenID to backend for process

// ViewController.swift

// Stripe-iOS

// Created by RANBIJAY KUMAR on 03/03/17.

import UIKit

import Stripe

class ViewController: UIViewController, STPAddCardViewControllerDelegate {

override func viewDidLoad() {

super.viewDidLoad()

// Do any additional setup after loading the view, typically from a nib.

}

@IBAction func actionAddCardDefault(_ sender: AnyObject) {

let addCardViewController = STPAddCardViewController()

addCardViewController.delegate = self

// STPAddCardViewController must be shown inside a UINavigationController.

let navigationController = UINavigationController(rootViewController: addCardViewController)

self.present(navigationController, animated: true, completion: nil)

}

// MARK: STPAddCardViewControllerDelegate

func addCardViewControllerDidCancel(_ addCardViewController: STPAddCardViewController) {

self.dismiss(animated: true, completion: nil)

}

func addCardViewController(_ addCardViewController: STPAddCardViewController, didCreateToken token: STPToken, completion: @escaping STPErrorBlock) {

//Send token to backend for process

print(token)

self.dismiss(animated: true, completion: {

completion(nil)

})

}

override func didReceiveMemoryWarning() {

super.didReceiveMemoryWarning()

// Dispose of any resources that can be recreated.

}

5. Create another Swift file i.e, StripeCardVC.swift and paste the following code inside it. This view allow users to enter detail of the card manually and getting tokenID to backend for process

// StripeCardVC.swift

// Stripe-iOS

// Created by RANBIJAY KUMAR on 03/03/17.

import UIKit

import Stripe

class StripeCardVC: UIViewController, STPPaymentCardTextFieldDelegate {

@IBOutlet weak var btnBuy: UIButton!

let cardParams = STPCardParams()

override func viewDidLoad() {

super.viewDidLoad()

let paymentField = STPPaymentCardTextField(frame: CGRect(x: 10, y: 100, width:self.view.frame.size.width - 20, height: 44))

paymentField.delegate = self

self.view.addSubview(paymentField)

// Do any additional setup after loading the view.

self.btnBuy.isEnabled = paymentField.isValid

btnBuy.backgroundColor = UIColor.gray

}

// MARK: STPPaymentCardTextFieldDelegate

func paymentCardTextFieldDidChange(_ textField: STPPaymentCardTextField) {

print("Card number: \(textField.cardParams.number) Exp Month: \(textField.cardParams.expMonth) Exp Year: \(textField.cardParams.expYear) CVC: \(textField.cardParams.cvc)")

self.btnBuy.isEnabled = textField.isValid

if btnBuy.isEnabled {

btnBuy.backgroundColor = UIColor.blue

cardParams.number = textField.cardParams.number

cardParams.expMonth = textField.cardParams.expMonth

cardParams.expYear = textField.cardParams.expYear

cardParams.cvc = textField.cardParams.cvc

}

@IBAction func actionGetStripeToken() {

STPAPIClient.shared().createToken(withCard: cardParams) { (token, error) in

if let error = error {

// show the error to the user

print(error)

self.showAlertButtonTapped("Error", strMessage: error.localizedDescription)

} else if let token = token {

//Send token to backend for process

print(token)

self.showAlertButtonTapped("Success", strMessage: "Got Token Successfully")

}

//MARK:- AlerViewController

func showAlertButtonTapped(_ strTitle:String, strMessage:String) {

// create the alert

let alert = UIAlertController(title: strTitle, message: strMessage, preferredStyle: UIAlertControllerStyle.alert)

// add an action (button)

alert.addAction(UIAlertAction(title: "OK", style: UIAlertActionStyle.default, handler: nil))

// show the alert

self.present(alert, animated: true, completion: nil)

}

@IBAction func actionBack(_ sender: AnyObject) {

self.navigationController?.popViewController(animated: true)

}

override func didReceiveMemoryWarning() {

super.didReceiveMemoryWarning()

// Dispose of any resources that can be recreated.

}

// MARK: - Navigation

// In a storyboard-based application, you will often want to do a little preparation before navigation

override func prepareForSegue(segue: UIStoryboardSegue, sender: AnyObject?) {

// Get the new view controller using segue.destinationViewController.

// Pass the selected object to the new view controller.

}

* Please be noted, Stripe are available for businesses in 25 countries, accept payments from anywhere in the world. Checkout the details here: https://stripe.com/global#signup-form

6. The complete working-code sample user can be fetched from Github. Please clone or download from here: https://github.com/RANBIJAY/Stripe-iOS

After downloading the project in local drive - open it in Xcode IDE. Few things need to be done before Build or Run the project:

Open ‘Build Settings’ of the project; search ‘Objective-C Bridging Header’ filed. Drag the file ‘Stripe-iOS-Bridging-Header.h’ file from Project navigator and put into the corresponding field of ‘Objective-C Bridging Header’. It will set the path of the file and needed to avoid linking error during runtime of the project.

Once set with the above sample in machine, build or run the app – you can see the tokenID been generated – shown in below screenshot

App Screenshots of multiple screens:

That's it for this blog.

Stay tune for next exciting blog .....!

Swift Programming (Server-Side) - Kick Start..!

2017-03-02T04:49:00.001-08:00

In this post we are going to learn some exciting stuff about Swift programming. Most of us are knowing or heard about the Swift, an additional programming language apart form Objective-C which been offered by ‘Apple Computers’ for developers and programmers to build the native app. Here we will learn quick basics about how to use Swift in Server-Side programming and play around with the web component methods or services and their integrations at high level. Also achieve bit idea of incorporating services with the cloud functionality.

Moving forward, certainly we need to set-up an environment first into the development machine. To do that, refer below links. Please be noted I did this sample in my Mac machine, optionally, it can be achievable for other platforms too –for that I encourage to check guidelines and detailed contents available under same link pages!

Important Links:

· https://swift.org/getting-started/#installing-swift

· http://perfect.org/docs/gettingStarted.html

· https://github.com/PerfectlySoft/Perfect

· https://github.com/PerfectlySoft/Perfect-HTTPServer

Set-up Environment:

· MacOS Sierra – 10.12.3

· XCode – 8.2.1

· Rested – 2.7

· Swift – 3.0.2

· Perfect - Server Side Swift [Open Source - powered by Swift Programming Language Framework]

-Perfect is an open source server side Swift framework that helps you write web apps and services using Swift 3. Learn how to get started with Perfect, and create your first server side Swift web app.

-Deploying Perfect in additional environments including Heroku, Amazon Web Services, Docker, Microsoft Azure, Google Cloud, IBM Bluemix CloudFoundry, and IBM Bluemix Docker – stay tuned for future post or refer above link documentation.

Now, lets start with the steps. Once you installed all above software and tools – mentioned under set-up environment. Open Terminal in Mac machine:

$ swift -- version

$ mkdir hello-perfect

$ cd hello-perfect

$ swift package init --type executable

$ swift package generate-xcodeproj

$ open ./hello-perfect.xcodeproj/

Update Package.swift file from here:

· https://github.com/RANBIJAY/Server-Side-Swift-Sample/blob/master/hello-perfect/Package.swift

$ swift package update

$ swift package generate-xcodeproj

$ mkdir webroot

$ touch webroot/hello.txt

Update hello.txt file from here:

· https://github.com/RANBIJAY/Server-Side-Swift-Sample/blob/master/hello-perfect/webroot/hello.txt

$ swift package generate-xcodeproj

Set the active scheme -> press Edit scheme -> check ‘Use custom working directory’ and pick the path of the ‘webroot’ folder which does have ‘hello.txt’ file inside before building the app or run it.

Copy or update main.swift file from here:

https://github.com/RANBIJAY/Server-Side-Swift-Sample/blob/master/hello-perfect/Sources/main.swift

$ swift package generate-xcodeproj

Check on console the status of HTTP Server – after building or running the app, should be started with the http port number (8080). Also, we can check into the browser by typing http://localhost:8080/. It will response the string message which been programmed under main.swift file. Alternatively, you can test the result of the app via ‘Rested tool’, which is available on Apple App store free to download. Below check the output response screenshot for visibility.

The purpose of this post to learn about how to kick start server-side programming skill using Perfect Open Source Framework (i.e., powered by Swift Programming Language) - using instance of HTTPServer API method calls such as, GET, POST and others. This skill will definitely help developers and programmers to use Swift language competency on both client and server side in parallel. Also developers can get benefits to develop native mobile applications in iOS platform without having additional language skills, the approach could be considered in hybrid method architecture too with the aspect of other platform choice.

To learn or explore more about advantages, benefits and possibility of the framework, I encourage to deep dive into the above-shared links, will be useful to kick start with. Thanks!

Alexa skills from scratch with AWS Lambda, SSML and Amazon S3

2017-02-18T11:06:00.000-08:00

Hello, everyone in this post I am going to teach about creating custom Alexa skills from scratch with AWS Lambda based code-functions and walk you through how to use SSML (Speech Synthesis Markup Language) and Amazon S3 inside the skills. As we have many geek code-snippets and webinar material available on cloud or web, but here I tried to cover all noted and important takeaway bullet points – help on creating sample skills in single flow without facing any obstacles or challenges in your development plus implementation phase. For mobile, cloud & IoT devs and professional, it’s a piece of sample case study, which offers them to accelerate their existing skills from basic to expert level.

Here I recommend to use below links for setup-environment and source management of Alexa skills development and testing of them:

A. First, install Node/NPM into your development machine. Make sure it’s updated with the latest version.

B. Amazon Alexa Skills Kit:

https://developer.amazon.com/

https://developer.amazon.com/alexa-skills-kit - Press ‘Alexa’ Tab to build an Alexa skills

The Intent Schema, Custom Slots, and Sample Utterances for this skill, as well as testing instructions are located at http://amzn.to/1LzFrj6

For additional samples, visit the Alexa Skills Kit (ASK) Getting Started guide at http://amzn.to/1LGWsLG

* Create an account first for accessing AWS cloud services.

C. Amazon Developer Console - Services:

https://console.aws.amazon.com/lambda/ - Lambda (Server-less Computing)
https://console.aws.amazon.com/s3 - S3

D. Echosim.io (Free Community Edition Beta) – Alexa Skill Testing Tool: In case you don’t have access of Amazon Echo device, Echosim.io cloud tool help you on developing/testing your custom skills:

https://echosim.io/

E. Audacity – to create mp3 files in specified format:

http://www.audacityteam.org/download/

http://manual.audacityteam.org/man/faq_installation_and_plug_ins.html#lame

Step1: Create an AWS account via Developer Console

Step2: Get Start With the Alexa Skills Kit (ASK)

Step3: Fill the skill-information, such as name, invocation name, ... press Next Get Start With the Alexa Skills Kit (ASK)

Step4: Create interaction model. Essentially all intent schema, slot types and possible utterances details and press Next

Step5: In Configuration page, select option as depicted below except text-field info (required ARN here to fill or linking). Make yourself pause in proceeding to Next button.

Step6: Open AWS Lambda service via 'Developer Console' and 'Select blueprint'. You can create your own, but for this sample I chosen via filter 'alexa' keyword - 'alexa-skills-kit-color-expert' sample blueprint.

Step7: Configure triggers in this phase. At this moment only pick the region 'US East (N. Virginia)' to create and test the skills freely. Currently APAC region doesn't allow to add a 'Alexa Skills Kit' ASK trigger that will invoke your function.

Step8: Follow as depicted below and mentioned in above step.

Step9: Follow here as mentioned in above step to invoke your function. Filter/Pick 'Alexa Skills Kit.

Step10: In this step, fill information of 'Configuration Information' page, such as function name, description and desired programming language from. Here I picked Node.js 4.3

Step11: After you provide the code of your function, fill Lambda function handler and role as depicted below in screenshot.

Step12: After you provide the code of your function, fill Lambda function handler and role as depicted below in screenshot. Press 'Actions' tab and choose 'Configure test event' option, then select 'Alexa Start Session'. Press 'Save and test' button to check the code function log and desired result outcome.

Step13: After you provide the code of your function, fill Lambda function handler and role as depicted below in screenshot. Here you'll get ARN - copy this one and paste in 'Step5' image and proceed for the Next (i.e., under Developer Console portal - created earlier).

Step14: Once the above steps been completed and achieved execution result:succeeded. Then time to test the real skills app. Copy the 'strings' as highlighted below and test the app via in-built simulator.

Step15: Developer can test the SSML based tag command under in-built 'Voice Simulator' option. use/modify SSML tags instead of 'Plaintext' inside the Lambda code function and hear the voice in response. SSML piece still on developing phase, currently support MPEG (.mp3) format in specified way. *Regarding how to create/record audio locally by using 'Audacity app' (i.e., link available above) and upload same content to Amazon S3 - will write separate post, stay tuned.

Step16: We can visualize the request and response in JSON format inside the window - depicted run-time sample in below screenshot.

Step17: Time to test the succeeded app response through Echo cloud device i.e., echosim.io. Press mike button on below screen and use 'utterances' to chat with Alexa Echo device. Please be noted the echosim.io use same credential of AWS developer console to react/response of programmed skills queries. Amazing one!

Sample Code Copy - Github : https://github.com/RANBIJAY/alexa-Skills-Samples

MEAN Setup and Start with @ Mac OSX..!

2016-12-28T08:54:00.001-08:00

Dear Viewer,

This post will help and take you with the following possible steps in regard of installing stack tools associated with the MEAN development platform into your Mac OSX machine. I have seen there are plenty of posts available online for setting up work environment on various cross-platforms. My focus is also quite similar over here; but this write-up is only targeted for Mac OSX users. Due to large volume API up-gradation and changes in technology cycles frequently many things might not go right in one short of your MEAN development set-up at Mac machine. With having said that, thought to write a post should be meaningful and helpful for target audience. I have been done setup implementation successfully in my Mac OS (i.e., macOS Sierra version 10.12.2). Lets start with the step of set-up implementation after getting bit idea/purpose of MEAN:

MEAN (Mongo, Express, Angular, Node) - A simple, scalable and easy starting point for full stack JavaScript web development. The MEAN stack represents a thoroughly modern approach to web development: one in which a single language (JavaScript) runs on every tier of your application, from client to server to persistence.

Here understand the short purpose and definition of all stacks:

• MongoDB is the leading NoSQL database, empowering businesses to be more agile and scalable.

• Express is a minimal and flexible node.js web application framework, providing a robust set of features for building single and multi-page, and hybrid web applications.

• AngularJS lets you extend HTML vocabulary for your application. The resulting environment is extraordinarily expressive, readable, and quick to develop.

• Node.js is a platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications.

Before you begin, you should make sure you have installed all these prerequisites on your development machine.

The first is to change the owner of “/usr/local” to the with the following command:

$ sudo chown -R $USER /usr/local

It is not recommended to use the npm command with sudo.

The second would be to install the Apple command line developer tools. I think, the best way to do it is by installing XCode. Yes you then have another big IDE, but it installs much more. But this way, you don’t need to have a developer account. I did it by Installing XCode; you can just install the command line tools. XCode can be installed from the Apple AppStore.

Step 1:

Install either Macports or HomeBrew in your Mac machine for easy installation of tools and their dependencies.

The third is to install homebrew. Homebrew is the package manager for MAC. The installation is pretty easy, type the following in you command line:

$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Then run the homebrew doctor like this:

$ brew doctor

After this, you can install tools like this:

$ brew install wget

Step 2: Installing Node.js / Nodes.js Package Manager (NPM)

Download & Install Node.js and the npm package manager, if you encounter any problems, you can also use this Github Gist to install Node.js. Again, we'll let HomeBrew do the heavy lifting:

$ brew install node

The npm executable is already included in the Node.js package.

Before continuing, let's make sure Node.js modules can be found by others (Caution: it's best to copy and paste these commands, as you'd lose the original contents of your .bashrc file if you typed > in place of >>):

$ echo 'export NODE_PATH="./node_modules:/usr/local/lib/node_modules"' >> ~/.bashrc && source ~/.bashrc

To check if the configuration is in effect, execute:

$ echo $NODE_PATH

You should see ./node_modules:/usr/local/lib/node_modules printed out below your command. If you use a different shell than Bash, simply replace ~/.bashrc with your shell configuration file.

Step 3: Installing MongoDB

http://docs.mongodb.org/manual/tutorial/install-mongodb-on-os-x/

If you haven't used HomeBrew before, simply execute the following command in a Terminal window:

$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)".

We will install MongoDB through howebrew. Make sure homebrew is up-to-date by executing this command:

$ brew update

Then lets install MongoDB with the following command:

$ brew install mongodb

To run MongoDB, we have to create de default data directory for our MongoDB by:

$ mkdir -p /data/db

The user monogd must have write access.

Run MongoDB on the default port (27017) with:

$ mongod

Also you can use below command:

$ brew update && brew install mongodb

Homebrew will automatically install all the dependencies for you.

Also install some other package managers:

Install Bower

You’re going to use the Bower Package Manager to manage your front-end packages, in order to install it make sure you’ve installed Node.js and npm, then install bower globally using npm:

$ npm install -g bower

Install Grunt

You’re going to use the Grunt Task Runner to automate your development process, in order to install it make sure you’ve installed Node.js and npm, then install grunt globally using npm:

$ npm install -g grunt-cli

Install Yeoman

MeanJS uses Yeoman, the web’s scaffolding tool for modern web apps.

Install Yeoman globally with:

$ npm install -g yo

Install MeanJS generator for Yeoman

$ npm install -g generator-meanjs

You are now ready to get started with the MEAN.JS generator. The generator will help you create a MEAN.JS application, a CRUD module, Angular Module, and other AngularJS/Express entities. We’ll begin with the application generator.

Step 4: Generating a Fullstack app

Make a directory for your Back End Project projects. Assuming your desktop is your de facto workspace:

$ mkdir ~/Desktop/Back End Projects && cd ~/Desktop/Back End Projects

Now that all the preparations are in place, it's time for the main event:

$ yo angular-fullstack

Step 5: Initializing local Git repository

Tell Git not to track your database:

$ echo "data" >> .gitignore

Turn the directory in which your application is located into a Git repository by running the following commands:

$ git init && git add . && git commit -am 'initial commit'

Step 6: Starting MongoDB

To start MongoDB for the first time in your app's directory, run the following commands in your terminal:

$ mkdir data && echo 'mongod --config /usr/local/etc/mongod.conf --dbpath=data --rest "$@" --httpinterface' > mongod.sh && chmod a+x mongod.sh && ./mongod.sh

From this point on you can simply start MongoDB by executing ./mongod.sh. A few things to note:

• The .conf file directs mongod to write messages to a log file instead of stdout. To view the log, run the following in a separate Terminal tab: less /usr/local/var/log/mongodb/mongo.log.

• Since we're not on Cloud9, we don't need the --nojournal option. Journaling lets you recover the database in case of a mongod crash.

• You have to make a clean database for each project. If you copied the data directory over from an earlier project, mongod will fail to start. If that's the case, just rm -rf data && mkdir data && ./mongod.sh.

Step 7: Application Generator

Installing full stack tools

$ npm install -g express yo grunt grunt-cli generator-angular-fullstack bower

The application generator will help you create a fresh copy of a MEAN.JS application in your working folder. To create your MEAN application, navigate to a new project folder, and then use yo to generate your application:

$ yo meanjs

The generator will ask you a few questions about your new application and will generate it for you. When the installation process is over, you will be able to use grunt to run your new MEAN application:

Open a new Terminal tab by pressing ⌘T, and run the following command:

$ grunt serve

Grunt should automatically open your new Angular site's index page as soon as it finishes starting up.

Now you should be able to follow the rest of the Challenge instructions to push to GitHub and Heroku. Just ignore the part about SSH key (#33-36) and replace ~/workspace with your app directory's path.

HTS - High Throughput Satellite [Adoption in Mobility Startegy]

2016-11-01T00:08:00.001-07:00

Today world is lying on mobile and Internet ecosystems together. The proliferation of web-based mobile applications necessitated mobile broadband access to the Internet, which rapidly became widely and sufficiently available to users of mobile devices, as long as they stayed within the service range of their appropriate terrestrial networks.

Lets think on the situation from consumer stand point when mobile users got on planes, ships and land-mobiles, the bandwidth frequency, communication and Internet experience was quite different. How consumer and enterprise industry approach and dealt with the situation? HTS (High Throughput Satellite) is one of the approaching solution, which could be considered to become a game changer for Internet users in the aeronautical, maritime, and land-mobile market segments. HTS are primarily deployed to provide broadband Internet access service (point-to-point) to regions un-served or under served by terrestrial technologies where they can deliver services comparable to terrestrial services in terms of pricing and bandwidth. Let's understand what's HTS first? As per Wikipedia, HTS is a classification for communications satellites that provide at least twice, though usually by a factor of 20 or more, the total throughput of a classic FSS satellite for the same amount of allocated orbital spectrum thus significantly reducing cost-per-bit.

The advent of high-throughput satellites (HTS) enables network service providers to offer a new generation of communications solutions. HTS systems combine the exceptional spectrum efficiency and performance of spot-beam antennas with ultra-wideband transponders to enable unprecedented levels of bandwidth and throughput. Each spot beam reuses frequencies in multiple carriers so that a single HTS spacecraft can provide five to ten times the capacity of traditional satellites. For the customer, this provides the potential to dramatically increase data rates, upwards of 100Mbps to a single site and improve application performance compared to traditional satellite based communications.

While many current HTS platforms were designed to serve the consumer broadband market, some are also offering services to government and enterprise markets, as well as to terrestrial cellular network operators who face growing demand for broadband backhaul to rural cell sites. For cellular backhaul, the reduced cost per bit of many HTS platforms creates a significantly more favorable economic model for wireless operators to use satellite for cellular voice and data backhaul. Some HTS platforms are designed primarily for the enterprise, telecom or maritime sectors. HTS can furthermore support point-to-multipoint applications and even broadcast services such as DTH distribution to relatively small geographic areas served by a single spot beam. However, in simple terms, HTS is making broadband access better, cheaper and faster.

Here we look at five main important components of HTS design strategy are:

Throughput - total transmitted per seconds
Efficiency - actually interdependent of throughput requirements
Coverage - the region being served in spacecraft and geographically
Architecture - network architecture {Star, Mesh, Loopback…}
Spectrum/Efficiency - frequency in consideration into the available bandwidth network and characteristics of the spectrum

In business terms, HTS will significantly reduce the Total Cost of Ownership (TCO) compared to yesteryear’s satellite broadband solutions in the mobile platforms space. More importantly, HTS broadband access is finally becoming competitive with its terrestrial counterparts in terms of cost and performance. But the good news is that HTS does not have to compete with terrestrial solutions in the air, sea and land mobile segments, it just needs to become as widely available as its terrestrial cousins and support comparable broadband connectivity in terms of throughput and cost.

Takeaway from experts based on experimental outcome result and existing case-study report:

· Next generation High Throughput Satellites include both Ku and Ka [Refer - below comparison image table].

· There are many different variations of technologies using the Ka-band frequencies.

· Ka is an appropriate technology for mass markets and non-industrial markets.

· Side-by-side technical and cost comparison of Ka and Ku HTS solutions shows significant technical and cost challenges with most Ka solutions for customers with high reliability and availability needs.

· HTS Ku platforms offer the best overall performance and value for operations for mission critical, maximum uptime networks.

Important links to explore the purpose and needs of HTS are in detail, such as:

https://www.linkedin.com/pulse/hts-game-changer-bandwidth-run-mobile-platforms-jack-nargundkar
https://en.wikipedia.org/wiki/High-throughput_satellite
http://www.thedigitalship.com/conferences/presentations/2014rotterdam/10_Claude_Rousseau-Research_Director-NSR-France.pdf
http://www.harriscaprock.com/downloads/HarrisCapRock_WhitePaper-Ka-Ku_Analysis.pdf
https://www.youtube.com/watch?v=XrN4dtcpO8k

Thanks you for your time on staying at this page. Please be free to advise/contribute/suggest below in comment section. Will add all valuable relevant input quickly. Thanks again and look forward!

MVC, MVP & MVVM - Architectural Patterns.

2016-02-08T04:07:00.001-08:00

MVC - Model–View–Controller (MVC) is a software architectural pattern mostly (but not exclusively) for implementing user interfaces on computers. Traditionally used for desktop graphical user interfaces (GUIs), this architecture has become extremely popular for designing web applications.

MVC - Framework Library: Early web MVC frameworks took a thin client approach that placed almost the entire model, view and controller logic on the server. This is still reflected in popular frameworks such as Ruby on Rails, Django, ASP.NET MVC, Spring MVC and Express. Client technologies have matured; frameworks such as AngularJS, EmberJS, JavaScriptMVC, SpineJS and Backbone have been created that allow the MVC components to execute partly on the client.

MVP - Model-View-Presenter (MVP) is a derivation of the model–view–controller (MVC) architectural pattern, and is used mostly for building user interfaces. MVP is a user interface architectural pattern engineered to facilitate automated unit testing and improve the separation of concerns in presentation logic:

the view is a passive interface that displays data (the model) and routes user commands (events) to the presenter to act upon that data.
the model is an interface defining the data to be displayed or otherwise acted upon in the user interface.
the presenter acts upon the model and the view. It retrieves data from repositories (the model), and formats it for display in the view.

MVP - Framework Library: Client Side- Riot.js, GWT whereas Server Side - Classic ASP.NET, JSP Servlets. In general implementation of MVP:

.Net Framework - Claymore, MVC# Framework, Web Client Software Factory, Evolution.Net MVP Framework, ASP.NET Web Forms Model-View-Presenter (MVP), Nucleo.NET, WinForms MVP
Java Framework - JavaFX, MVP4J, Echo2, Google Web Toolkit,GWT-Platform, JFace, Swing, Vaadin, ZK
PHP Framework - Nette Framework

MVVM - Model–View–ViewModel (MVVM) is a software architectural pattern. MVVM and Presentation Model both derive from the model–view–controller pattern (MVC). MVVM facilitates a separation of development of the graphical user interface (either as markup language or GUI code) from development of the business logic or back-end logic (the data model). Model–view–viewmodel is called model–view–binder, especially in implementations not involving the .NET platform. ZK (a web application framework written in Java) and KnockoutJS (a JavaScript library) use model–view–binder. The MVVM pattern includes three key parts:

Model (Business rule, data access, model classes)
View (User interface (XAML))
ViewModel (Agent or middle man between view and model)

MVVM - Framework Library: Client side - Knockout.js, Kendo (MVVM) whereas Server side - WPF (Desktop) or Silverlight, Windows Phone apps (XAML), Adobe Flex.

When to use which?

MVC - Use in situations where the connection between the view and the rest of theprogram is not always available (and you can’t effectively employ MVVM or MVP)
MVP - Use in situations where binding via a datacontext is not possible.
MVVM - Use in situations where binding via a datacontext is possible.

Both MVP and MVVM are derivatives of MVC (see timelines and how these have evolved). The key difference between them is the dependency each layer has on other layers as well as how tightly bound they are to each other.

Developers concern and take on against Parse (MBaaS) cloud service shut down.

2016-02-03T05:14:00.000-08:00

In this blog we are going to discuss about the challenge and alternate approach of an initiative or best practice adoption after shutting down the Parse service i.e, mobile backend as a service (MBaaS).

As we all know that Facebook has made an announcement about shutting down the Parse cloud service. Currently it's not closed, it will stick around till up to the time of next year i.e., 28 Jan 2017.

Facebook Parse chief technology officer Kevin Lacker wrote in a blog post, "We’re proud that we’ve been able to help so many of you build great mobile apps, but we need to focus our resources elsewhere".

Challenge:

After Parse shut down, developer must look elsewhere,

Migration in new environment from existing one,

Dealing and convincing to the existing customers/clients, costing, manpower.. so on

Growing backend and architecture services team, etc.

Alternative Solution Approach:

Before we move on towards new alternate adoption it is good to analyze about what features are we looking for ? The list of services are:

Data Storage
Push Notifications
Usage Analytics
Dashboard
Social Integration
User Administration
Cloud Code Integration
Multiple Mobile Platform SDKs
Background Jobs
Crash Report

The 'Github Development Community' step ahead of this challenge and a great list of Parse alternatives has already emerged as a result.

In next what developers can do ?

Migrate and Host Own Parse Server: Parse has provided a Migration Guide (https://parse.com/docs/server/guide#migrating). Facebook is open-sourcing Parse Server and making available a database migration tool to help move apps off of Parse, that lets you migrate data from Parse app to any MongoDB database. Developers will run most of the Parse API from their own Node.js servers (or one hosted on a platform like Heroku).
Switch to Another SaaS: Developers need to update their application(s) to leverage new SaaS environment as its backend service, this could also be a great time to upgrade app(s) with improved or added functionality.
Build Own Backend Solution: SaaS platforms are great for MVPs, rapid prototyping, personal projects, and powering smaller features of larger products. If your application needs to be built to scale, is mission critical, and continues to evolve with custom needs, it may be a better business strategy to build your own backend. This way you maintain complete control and can leverage it as a competitive advantage.

The difference with Parse is that it has attracted more developers than the existing platforms because of its easy-to-use and well-documented SDKs, highly competitive (often free) pricing, intuitive backend portal, and its ability to handle everything (e.g. cloud computing, push services, user authentication, etc.) with very little configuration or setup. From big-name brands to indie shops, Parse was the obvious choice to accelerate production of MVPs, small to medium scale applications, or specific features of large scale applications. SaaS systems are often ideal for prototyping and personal projects, but for large, custom applications that company environment depends on, building own server stack is worth the extra time and effort. Parse isn’t the only player in the SaaS (Software as a Service) industry. Amazon, Google and Microsoft (and numerous other startups) now offer very similar tools for developers. The idea behind using a service like Parse, however, has always been that developers could avoid dealing with servers, so it remains to be seen how many of its current users will make this switch.

Commercially available things !

2016-01-18T04:21:00.001-08:00

There are many standards and proprietary solutions used for connecting things to each other or to the cloud: Wi-Fi, Bluetooth, ZigBee, Active RFID, loWPAN, EtherCAT, NFC, RFID to name a few. Among these technologies, Wi-Fi has been the most successful. It has become a ubiquitous standard of connectivity and is used in the home, enterprise, schools, hospitals, airports etc.

In this post, we will see some existing commercial things that we can use right now. These include

Smart home devices,

Medical devices,

Wearables, and

All kinds of gadgets.

All of these devices connect to your Android phone using either WiFi or Bluetooth, pacifically Bluetooth low energy (BLE), sometimes know as Bluetooth smart.

One of the most well-known internet of things (IoT) is the Phillips Hue Lighting System. talk to both Android Phone and Android wear watch.

Let's go take a look at some existing devices. Kaptur band this is a simple wrist band, that essentially consist of a microphone sensor, some memory, and a Bluetooth radio. It sits and records in a loop, 60 seconds, and whenever you tap it, it captures that 60 seconds, and sends it to your device.

Another well-known device is the Nest thermostat, which as I believe everybody know, is also owned by Google. They also make a smoke detector, both of which connect up via WiFi, and talk to your Android phone.

One of the most well-known companies in the home automation market is a company called SmartThings. They make a number of devices that allow you to connect up to your phone. They build locks, sensors, so you can tell when doors are opened or closed, cameras, speakers, and so forth.

Talking about the internet of things would be completely empty if we didn't talk about Android wear. The two most prominent devices right now for Android wear are the Moto 360 and the LG G watch. There's also the Pebble watch. It's not a specific product that's just for Android, but it does have a very nice SDK, and it actually has some physical buttons on it, so you can build apps that do a variety of things with the user punching the buttons on the device. Then, using your Android phone to either control your phone or to use your phone then via WiFi to control other devices.

Lumo Lift is a small device to track your activity and calories burned, as well as improve your posture. It is essentially an accelerometer, and you can see here it goes on the shirt/bra by holding it with a magnet. Essentially, you just sit up straight, double-tap it, and then if you ever start to slump, it'll buzz, vibrate and remind you to sit up straight again.

In terms of wearables and fineness, one of the most popular's called Fitbit. Fitbit, essentially, will tell you how many steps you've taken, route your map, and do all of that while you just wear it around everyday, and it'll help you manage your weight, eat better, sleep better, and so forth.

One of the early entries in the internet of things was a company called Withings. They make everything from activity tracker to a bathroom scales, to this fashionable Activite steel white watch and activity tracker. They also make a pulse oximeter, so that you can tell how much oxygen is in your blood while you're exercising, and even a home camera.

The internet of things goes everywhere, including into your peaceful moments. The muse, the brain sensing headband monitors your brain waves, and allows you to determine when you brain is actually calm. There's also an SDK that you can use to experiment with using your brain waves to control other things.

TI Sensor Tag. The reason it's great is because it's $25, and it contains all of these things, temperature sensor, humidity sensor, pressure sensor, accelerometer, gyroscope, and magnetometer. That works on one CR2032 battery, and it's very, very easy to talk to it with Bluetooth. And also you can do your Android phone or Android tablet to talk to a Bluetooth device. This is a easy way to get started, so that you can understand how to build a Bluetooth app.

Then, there's this great product that's a internet of things developer kit called WunderBar. It's from Germany from a company called Relayer. We'll talk to the Relayer master module via WiFi, and it will talk to the different individual modules via Bluetooth.

Then, we're going to go even further, and build a simple intelligent device that you can mount at your door, so that when somebody rings the doorbell, you can pick up your phone and snap a picture of who's there, so you can see who's at the door. For that, we're going to use the Tessel platform. Tessel's very interesting. It's a microcontroller that runs JavaScript. It uses no JS, has WiFi, it's got four connectors. Tessel's very unique in it, all of the components are only on one side. You can always tell when you've got things plugged in correctly with the Tessel, because it can only go in one direction. With components on one side, if you flip it over, all you see are the backs of all the boards. You can plug pretty much any component into any one of those things, and you can power the Tessel with via USB or with a battery. It's very, very easy to program using JavaScript.

Lot more are available or existing which can be used as a solution in demanding market.

Where to find new Internet of Things (IoT) Devices ?

2016-01-18T02:57:00.000-08:00

Hey folks,

If you are a crazy and passionate gigs for exploring Internet of Things (IoT) technology invention and gadget availability around. Also very fond to know about latest news, articles, pricing and purpose for various available sensors and IoT based gadgets details.

I recommend to checkout below links, will help on making decision and pursuing further:

Thank You!

Custom ROM: CM and MIUI - High Level Overview.

2016-01-05T10:21:00.000-08:00

Hello Readers,

Cheers and best wishes for 2016 :-)

With this post you'll grasp better knowledge and understanding about CyanogenMod (CM) and MIUI
(which stands for Mi User Interface and pronounced "Me You I", a play on the common abbreviation of the words user interface as UI).

Lets discuss first, what exactly the purpose to know about them?

There are list of custom ROM available in market as an open source. CM and MIUI are most popular which can be used on almost any phone brand and has support for custom ROMs. Both are an open source operating system for smartphones and tablet computers, based on the Android mobile platform. Developed as free and open source software based on the official releases of Android by Google, with added original and third-party code. Both can be used as firmware for smartphones and tablet computers based on the free software Android OS. It is similar to installing any OS on a PC.

If you love to play with technology and interested to buy a new Android based phone or flashing an existing phone with one of these ROMs or UIs theme, this article will help you to make decision and also to pick one among these two MIUI and CM.

Here is a short background comparison of the two:

MIUI, developed by Xiaomi Tech whereas CM is developed by CyanogenMod Open-Source Community as free and open source software based on the official releases of Android by Google, with added original and third-party code. It is based on a rolling release development model and quite similar to stock Android.

The source model of MIUI is an open source with closed source component whereas CM is an open source.

Both ROMs have version history with the date of their launching and criticism/controversies involved time-to-time , the recent version of MIUI is MIUI v7 and CM is CM v13.

Interesting to know that the original MIUI ROMs were based on Android 2.2.x Froyo and CM v6 sources. Xiaomi added a number of apps to enhance the basic framework; those include Notes, Backup, Music, and Gallery. Other than supporting own brand Xiaomi also offers MIUI to be flashed on other smartphone brands such as Samsung, Sony, HTC, LG, One Plus and Nexus. On the other side, Cyanogen OS 12, a variant of CM v12 for the OnePlus One and Yu Yureka was released in April 2015. Yu Yuphoria got Cyanogen OS 12 out-of-the-box when it was launched in May 2015. Cyanogen commercially develops operating systems pre-installed on some devices (OnePlus One, YU Yureka, YU Yuphoria, Andromax Q, Lenovo ZUK Z1, Wileyfox Swift, Wileyfox Storm) based upon the CM source code.

MIUI do have regional restriction on package manager side. It doesn't ship Google Play Services in Mainland China. Global versions of MIUI are been certified by Google. Also during the development of MIUI v4, Xiaomi started to remove Google services from their ROMs in order to build and monopolize its own environment of Xiaomi, including cloud services, paid themes and games (featuring MiCredit currency). Later, during development of MIUI v5, nearly all Google services had been removed from the Chinese versions.

On performance perspective,

Features supported by CM include native theming support, FLAC audio codec support, a large Access Point Name list, an OpenVPN client, an enhanced reboot menu, support for Wi-Fi, Bluetooth, and USB tethering, CPU overclocking and other performance enhancements, soft buttons and other "tablet tweaks", toggles in the notification pull-down (such as wi-fi, Bluetooth and GPS), app permissions management, as well as other interface enhancements. CM does not contain spyware or bloatware. In many cases, CM may increase performance and reliability compared with official firmware releases.

MIUI, the interface is similar to and competes with iOS's UI and Samsung's TouchWiz UI. MIUI has really taken the android community apart by the really amazing features that the ROM is providing to its end users. It features a heavily-modified user interface that does away with the Android app drawer and has drawn comparisons with Apple’s iOS. IT features additional functionality that is not found in the stock Android applications which include toggles on the notification pull down, new music, gallery, and camera apps, and an altered phone dialer that displays matching contacts as a user enters a number. The modified operating system has been criticized to be similar to the Apple’s iOS but that does not really matter for the end users as long as they are enjoying using it.

Source: http://en.miui.com/data/attachment/forum/201410/05/202651rxdbgqd8b8dlqza2.jpg.thumb.jpg

Experts take on as conclusion: pick MIUI for user-friendly features and intuitiveness; go for CM if you want to control every aspect or built-in features of your phone and want more controls with high stability; and go for stock Android for performance, stability and simplicity. Also you can find experienced user view over here.

Mobilizing .aspx web based desktop app on BlackBerry10 platform over BES10

2014-07-02T12:17:00.001-07:00

This blog gives a quick sense to understand the approach of problem statement i.e., mobilizing .aspx web based desktop application on BlackBerry10 and other major platforms significantly. To achieve the result, please verify the system requirements and make sure all setup should be done early before moving ahead.

For .aspx web based app: Visual Studio IDE is needed on your development machine. For creating the fresh project or existing project execution, Please follow following steps:

On Visual Studio IDE- > click on File Menu-> New Project -> Pick the recent template as ‘Visual C#’ and select ‘ASP.NET Web Application’ for executing the project. See below image for ref (Fig1)

Fig1. Running ASP.NET web app on Visual Studio IDE

Click on the ‘Release’ button. App will run and display on default browser with picked port number, like this http://127.0.0.1:48667/Web.UI/EmpLogin.aspx

Fig2. Running Web Application on Desktop Browser with Port value

Write click on the Solution Explorer -> Publish Website

Fig3. Actual path of the Application

Copy the path C:\PFC\Sample App\PrecompiledWeb\Web.UI

Active the IIS Sever on Desktop platform first. Then go to

Windows -> Start - > ctrl + R -> inetmgr

IIS Manger window will open, then click on Sites - > Default Web Site - > Right-Click and choose ‘Add Application’ option

Provide the application ‘Alias name’ and actual path i.e., same target location which received from Publish Web Site on Visual Studio IDE.

Right-Click on the application Alias -> Manage Application -> Browse

The app will open on the default browser, without port-number, i.e., http://127.0.0.1/PFC/

Fig4. Running Web Application on Desktop Browser without Port value

In next step, as we know BlackBerry Enterprise Service (BES 10) environment is setup on the dev environment.Target platform device has to be enabled with the BES services, such as, currently supports available for BlackBerry 10, iOS and Android.

Open the Workspace (BlackBerry 10)/Secure Workspace browser (iOS and Android), type the URL with the IP address and alias name, i.e., http://127.0.0.1/PFC/ (have been created above as and application alias) .

App will start showing on the device workspace browser:

Fig5. Running Desktop Web Application on BlackBerry 10 Secure Browser

For launching the same output as an application, there are following ways:

1. Hybrid approach – create a native app and call the application URL on web associated API in that.
2. Web app – simply call the URL at respective web view API. That’s it.

After creation of the app, it has to be pushed on Workspace (BlackBerry 10)/Secure workspace (iOS and Android).

If the app doesn’t have a target to implement specific features such as (push notification, other api, so on) on respective app on specific platform, then above solution would be the right approach to pick or go forward.

To get to know more about BES 10, please follow: BES

*****END*****

Solutions Methodology of Mobile App Testing on Different Platforms/Devices into the Cloud, Automation and Manual environment.

2012-07-13T06:28:00.000-07:00

Hi,

As we know the "Testing" phase play very important role into the SDLC process especially to improve the Quality and standard of the products from different-different corners. Since, the current generation of IT (i.e., enterprise and consumer organization) is moving towards into the mobile computing, day-by-day launching new kind of devices for different-different platforms with various kind of features makes little challenge for the tester to test the app (i.e., mobile app) into the several devices. In this post, I am going to highlight the standard approach and methodology of testing the application for different device platforms on top of their internal features and functionality. Nowadays, the mobile market has 4 main OS (Android, iOS, Blackberry and Windows Phone 7) that are applied to a wide range of handsets. This implied uniformity in the way an application looks across all devices that run the same OS. So, for example, an Android application would look practically identical on any Android mobile phone. Therefore, an application vendor can create a test on one android device and run – the very same test – on all Android devices (phone, tablet etc).

Here, I am introducing and highlight the existing solutions which is mainly used as the powerful mobile automation tools. They are:

PerfectoMobile,
Keynote DeviceAnywhere,
SeeTest by ExperiTest,
FoneMonkey (MonkeyTalk), and
JAMO, etc.

*Note: Except of the FoneMonkey (MonkeyTalk) - all others are commercial tools and not free.

1. PerfectoMobile: Perfecto Mobile is a global leading provider of cloud-based testing, automation and monitoring solutions for mobile applications and websites, utilizing a wide selection of REAL mobile devices accessible via the web.
The MobileCloud enables developers and testers, located anywhere in the world, to access a comprehensive range of the latest mobile handsets and tablets via the Internet. Users can develop, test, deploy and monitor their mobile applications and services without having to physically obtain the handsets. The MobileCloud handsets and tablets are connected to LIVE cellular networks.

Perfecto Mobile solutions are specifically tailored to address mobility in the various stages of the application life cycle and to be used by the various departments.

Enterprise Class Mobile Testing Solution:

Automate - test cases and re-use them across multiple devices
Test new smartphones & tablets as soon as they reach the market
Share & collaborate with globally distributed teams and partners
Integrate with existing ALM systems
Constantly monitor performance & availability on real handsets

1.1 MobileCloud for QTP

A fully integrated mobile testing solution for Quicktest Professional

MobileCloud for QTP is a complete solution for automated functional and regression testing of mobile applications within HP's QuickTest Professional (QTP) product. MobileCloud for QTP seamlessly integrates native QuickTest Professional scripting and flow control with Perfecto Mobile's comprehensive cloud-based automated mobile testing solution. This breakthrough solution enables you to naturally extend your existing HP Functional Testing and Application Lifecycle Management (ALM) environment to mobile applications.

Why Use MobileCloud for QTP?

Innovative design and natural integration with QTP enable you to cost-effectively extend your functional and regression testing environment to mobile:

Native QTP scripting for mobile - No need to learn new scripting language facilitates script creation and maintenance
Fully integrated user interface - Use the same UI for normal QTP scripts, mobile scripting and hybrid scripting - Screenshots and video improve collaboration among globally distributed teams and shortens correction cycles
Rich image-enhanced reports - Single centralized test management platform maximizes cross-organizational visibility and reduces TCO
Seamless integration with HP Quality Center ALM - Single centralized test management platform maximizes cross-organizational visibility and reduces TCO
Cloud-based access to REAL handsets in live networks - Improved quality without the hassle of procurement and network allocation
Device-agnostic mobile scripting - Build test cases that can be easily maintained, reused and ported to multiple mobile platforms

The Best in Cloud-Based Mobile Testing - Inside QTP!

MobileCloud for QTP lets you enjoy the best of both worlds – you get the world's most advanced mobile test automation solution, while preserving your existing ALM frameworks and methodologies. Key features and capabilities include:

Secure cloud-based access to real devices - testers can select and immediately access hundreds of mobile devices from inside QTP.
Fast, easy and maintainable mobile scripting – testers can use high-level intelligent mobile-specific functions that emulate real user operations (e.g., press keys, open browser, select image, type text, etc.)
Device-agnostic keyword-based script creation - ScriptOnce technology enables keyword-based scripting, dynamic search for text and images, scrolling and visual relationship association.
Tight UI integration - users work on the device directly from the familiar QTP UI, using the same QTP feature set and scripting environment for normal QTP scripts, mobile scripting and hybrid scripting.
Rich image-enhanced reports - complete testing information, include active screenshots and video recording, is embedded into the QTP report.
Full back-end integration with HP Quality Center – Native QuickTest Professional scripting, data and report structures enable centralized test management and cross-organizational visibility of your mobile apps via HP Quality Center.

1.2 MobileCloud-Interactive

The MobileCloud-Interactive service enables users to test mobile applications, websites or services on a multitude of real handsets and tablets via the web and control them as if holding them in their own hands. Using MobileCloud-Interactive, you'll be able to streamline development, testing and support processes, perform more tests and ensure optimal compatibility of your product or service, on more handsets, networks and locations.

Main Features

Directly install applications from your PC to the device
Send text and numbers to the device using your keyboard
Simulate handset events in a single click (Call Me and SMS Me)
Capture screenshots and record videos
Share test reports with other colleagues and partners
Share devices with others to demo new apps/features, reproduce a bug
Reserve handsets for use whenever you need

1.3 MobileCloud-Automation

MobileCloud-Automation allows users to execute automated testing on real handsets and tablets located anywhere in the world. Users can automate their testing using the intuitive web-based user interface, which allows the creation of keyword-based ScriptOnce scripts. Once the scripts are done, users anywhere in the world can access the devices and run the scripts.

Benefits

Easy and efficient script creation – Reduced manual testing efforts on all platforms
Full flow control – including conditions, loops, flow commands, expressions and data-tables
Detailed reports - Reporting of each execution includes vast variety of information such as actual results, screen captures in digital format, continuous video recording of the entire script and graphs showing statistics about the execution
Enhanced collaboration - saves time and money on problem reproduction by using real time online handset sharing with multiple participates
Open REST API - allowing the integration into 3rd party tools such as test management, monitoring, defect tracking and ALM in general.

Benefits at a Glance

Fastest Time to Publication – Pre-certification and automated testing drastically reduce queues and rejects, resulting in huge time savings (up to 75%). Thus, premium applications are available in the store as soon as a new handset is launched
Increased quality and reduced risk – Automated testing and monitoring means you can test on many handsets, ensuring the highest levels of quality and user experience
Gain a competitive advantage – Simplifying the certification process will attract more developers to your store, enhance your reputation and reduce operational costs
Cost-effective - Automation lets you perform more tests on more handsets, locations and networks in less time and with less effort from your testing team, and less cost
Flexibility - Handsets can be added or replaced within a matter of hours with only a simple software update, enabling app stores to quickly respond to market trends

2. Keynote DeviceAnywhere: Provides a service for planning, testing and monitoring the functionality, usability, performance and availability of mobile apps and websites.

2.1 Enterprise-Class Mobile App Lifecycle Management (mALM):

Keynote DeviceAnywhere’s Test Center Enterprise (TCE) cloud includes interactive manual testing, test automation, and monitoring solutions allowing companies to bring higher quality mobile apps and websites to market faster, at a lower cost. TCE also provides intuitive planning, collaboration, management and reporting tools enabling customers to manage their mobile app lifecycles completely in the cloud.

Cloud-Based, SaaS Architecture

24x7, remote online access to all popular mobile devices including Android, iOS, BlackBerry, Windows Phone 7, and WebOS Smartphones and Tablets.
An always-on, globally accessible device bank saves significant cost and effort vs. establishing and managing mobile QA labs for distributed teams.
Test and monitor across multiple devices or test device-to-device interactions from any location in the world.
Online reporting (including video and screen capture) and collaboration enable efficient review and sharing with colleagues.
Built for on-demand use in our enterprise cloud environment as well as for private cloud deployments.

100% Mobile Ready – No Restrictions

Test or monitor any mobile app or website – native, HTML5, network-aware, streaming, multi-media, etc.
No infrastructure investment required.

Overview:

2.1.1 Planning: Keynote DeviceAnywhere helps enterprises target their development and test efforts on the most relevant devices and platforms with two free, valuable tools:
- Device Planner is a tool aimed at helping developers and businesses decide which devices and platforms to develop on, to reach the largest potential audience for their app.
- Test Planner is a tool which provides companies with a recommended list of devices for testing their apps on, to generate optimal test coverage using the least resources.

2.1.2 Execution:
- Test Center Enterprise Automation is a SaaS enterprise software service that automates testing of mobile apps and websites.The product aims to ensure reliable, repeatable and reportable results for any mobile app, on any device, operating system and network. Using the service, engineers can perform unit testing on real devices. Similarly, testers are able to troubleshoot field issues and perform the following kinds of tests: UAT; BAT; Functional Testing; Regression Testing; and Integration Testing.
- Test Center Enterprise Interactive is a cloud-based service designed for manual testing of mobile apps, designed for enterprise users. The service allows enterprises to remotely interact with and control a device’s functions including pressing handset buttons, viewing LCD displays, listening to ringers and tones, playing videos, tapping and swiping touch screens, muting, powering on/off, increasing volume and other functions as if they were holding the physical device in their own hands.
- Test Center is a cloud-based service designed for manual testing of consumer-facing mobile apps.
2.1.3 Analysis:

Test Center Enterprise Monitoring is a SaaS platform targeted at enterprises, which provides monitoring of mobile services to detect potential problems early and ensure quality of service.

3. SeeTest by ExperiTest: Experitest’s SeeTest product suite is based on a patented technology that uses self-learning diagnostic and matching algorithms and a modular self- enhancing image recognition technology.SeeTest uses a client-agent architecture and plugs into all existing test automation environments, such as QTP, C#, JUnit, Python, and Perl.

ExperiTest does have 3 major services:

Mobile Automation:Automation tool. Records on real devices. And on Emulators. Plugs into QTP, TestComplete, C#, RFT, Java, Perl, Python.
Mobile Manual:Manual testing tool. View the mobile device on your computer screen & test using your computer mouse & keyboard.
Mobile Manual:The mobile device remote access & scheduling platform.

Features

Connect all mobile devices to one central hub/server that resides in your organization
Enable remote access from anywhere in the organization to any device
Book & Schedule devices
Advanced booking options including: device filter by OS, SW version, tablet/device and more
Enforce device usage and remote access based on user class

Benefits

Top security – because it is your own organization’s security
Easy remote access to a large inventory of mobile devices
Efficient usage of mobile devices for testing by sharing resources (by accessing remotely) across the organization
Support for any device, including new devices and proprietary hardware devices that have not yet been commercially launched

4. FoneMonkey (MonkeyTalk):
It's free & open source functional testing tool for iOS and Android Mobile Applications

MonkeyTalk is the leading tool for automated testing of iOS and Android applications
Robust, cross-platform recording and playback
Test native, web, or hybrid apps, on iOS simulators, Android emulators or real devices -- no jailbreaking required
Everything from simple "smoke tests" to sophisticated data-driven test suites
Another great free and open source functional testing tool from Gorilla Logic

For more details: Please refer below link to understand the services into video format.

http://vimeo.com/monkeytalk/part1

Other features include:

Robust script recording and playback – more comprehensive recording and playback for user interactions, including taps, keyboard input, dragging and scrolling for all Cocoa Touch components.
Code generation options – FoneMonkey can now generate ready-to-run test scripts in Objective-C that can be run under OCUnit, or JavaScript that can be run with Apple’s UI Automation tools.
Optimized for both iPhone and iPad platforms and user interfaces.
Works on both simulators and real devices.
Test playbacks can be completely automated.

Please refer "QuickStart demonstrates using FoneMonkey" and "Setup process with different versions of XCode" via hlinks and their respective screenshots and videos.

5. JAMO: Jamo Solutions automates testing of mobile applications by using the same techniques on mobile devices that made automated testing on the PC so successful. Using the Jamo solutions M-eux Test software will be able to replay the actions of the end user directly on the device.

M-eux Test

The M-eux Test software allows you to automate, monitor and maintain your mobile test cases. M-eux Test recognizes the GUI elements of your mobile application resulting in test cases that are :

Device model independent.
Completely re-useable across different versions of the service/application under test.

M-eux test integrates flawlessly with existing well known environments such as Eclipse, QuickTest Professional from HP and Visual Studio from Microsoft. This approach results in scripts that are :

Easy to create and maintain

Architecture:An agent is installed on the mobile device. The agent is responsible for the recording and the replay of the test commands The Device is connected to the PC using either WIFI or a USB connection. The Device Manager is the gateway between the connected devices and the extension inside Visual Studio. The architecture allows the connection of multiple devices. One test script can execute against multiple devices.

Supported Mobile OS:

Device Operating System	System Supported versions
Android	Android 1.5 Android 1.6 Android 2.0 Android 2.0.1 Android 2.1 Android 2.2 Android 2.3 Android 2.3.3 Android 3.0 Android 3.1 Android 3.2 Android 4.0.1 Android 4.0.3 Android 4.0.4
BlackBerry	BlackBerry OS 4.6 BlackBerry OS 4.7 BlackBerry OS 5.0 BlackBerry OS 6.0
iOS (iPhone, iPad and iTouch)	iOS 4.2.x iOS 4.3.x iOS 5.0.1 iOS 5.1 iOS 5.1.1
Windows Mobile	Windows Mobile 2003 Pocket PC second edition Windows Mobile 2005 Pocket PC Windows Mobile 6.0, 6.1 and 6.5 Classic, Standard & Professional
Windows CE	Windows CE 5.0 Windows CE 6.0
Windows Phone	7.5

Here users can get the lists of sample videos for the demo of the tools and it's methodology:

" Video Sample "

Address Security concern in general with the Enterprise Mobility CoE

2012-06-29T05:13:00.000-07:00

Hi,

As we know Enterprise Mobility CoE continuously tracks technology developments in this space to develop offerings that address specific needs of their different-different verticals customers. It’s the age of bring-your-own-device (BYOD) and I.T. security professionals need a proven way to enforce and extend identity governance from the enterprise across a plethora of mobile devices. The security challenges into the Mobility CoE trends pose for customers and address them with the following competencies:

1. Integrating login authentication with customers existing active directory server

2. Implementing Two-Factor / Two-Channel Authentication. Some of recommended solutions are below:

Verification through SMS
Validating DeviceID along with UserID
Integrating "RSA Secure ID" services
Integrating with third party seamless services such as SecureAuth, PhoneFactors, etc.

3. Data encryption & compression for offline data storage

4. Integrated admin tools that will provide unified admin experience for different mobile

platforms.

5. Identify Management (IdM)

6. Single Sign-on (SSO)

7. Security Token Services

Difference between MDM and MAM

2012-06-19T00:35:00.000-07:00

Hi Friends,

I am going to highlight about the Mobile Device Management (MDM) and Mobile Application Management (MAM) concepts in this blog. First we have to understand what exactly these concepts do and how it'll be useful for the IT managements environment. MAM is very different from MDM. As we know, devices are made to be manage by heavy handed profiles into the IT to get mainly corporate information security and protection of data to the device level. That is mean if anyone lost his device , they can wipe up it from the remote. Application is management in mobile application a really meant for mobile workforce. This is what we get the envy and come up with the great ideas of apps that empowered us , and we need to start with the platform really knows how to do this in four life cycle:

Environment from upfront
Injection of policies
Security Wrapping
Right through the deployment and management.

Apart from the above points, using two very important things which we need to take care , i.e, user experience, no training required , getting the apps securely. And into the IT side, all over they need to do this from secure environment and they can do across a large enterprise that scales mobile application management and mobile device management.

Lets understand the standard and brief definition of MDM and MAM:

Mobile Device Management (MDM): MDM software secures, monitors, manages and supports mobile devices deployed across mobile operators, service providers and enterprises. MDM functionality typically includes over-the-air distribution of applications, data and configuration settings for all types of mobile devices, including mobile phones, smartphones, tablet computers, ruggedized mobile computers, mobile printers, mobile POS devices, etc. This applies to both company-owned and employee-owned (BYOD) devices across the enterprise or mobile devices owned by consumers.

By controlling and protecting the data and configuration settings for all mobile devices in the network, MDM can reduce support costs and business risks. The intent of MDM is to optimize the functionality and security of a mobile communications network while minimizing cost and downtime. With mobile devices becoming ubiquitous and applications flooding the market, mobile monitoring is growing in importance. Numerous vendors help mobile device manufacturers, content portals and developers, test and monitor the delivery of their mobile content, applications and services.

Implementation:

Typically solutions include a server component, which sends out the management commands to the mobile devices, and a client component, which runs on the handset and receives and implements the management commands. In some cases,

a single vendor may provide both the client and the server,
in others client and server will come from different sources.

One of the next steps was to allow a client initiated update, similar to when a user requests a Windows Update.

Central remote management, using commands sent over the air, is the next step. An administrator at the mobile operator, an enterprise IT data center or a handset OEM can use an administrative console to update or configure any one handset, group or groups of handsets. This provides scalability benefits particularly useful when the fleet of managed devices is large in size.

Central functionality includes:

FOTA - Firmware over the air updates
Diagnostics
Remote Configuration and Provisioning
Security
Backup/Restore
Network Usage and Support
Server Deployment
Mobile asset tracking and management
Remote Lock and Wipe
Device Provisioning
Software Installation
Troubleshooting and Diagnostic Tools
Policy Application
Logging and Reporting
Remote Control and Administration
GPS tracking and 'breadcrumb' mapping

Over-the-air programming (OTA) capabilities are considered a main component of mobile network operator and enterprise-grade Mobile Device Management software. These include the ability to remotely configure a single mobile device, an entire fleet of mobile devices or any IT-defined set of mobile devices; send software and OS updates; remotely lock and wipe a device, which protects the data stored on the device when it is lost or stolen; and remote troubleshooting. OTA commands are sent as a binary SMS message. Binary SMS is a message including binary data.

Mobile Application Management (MAM): MAM describes software and services that accelerate and simplify the creation of internally developed or “in-house” enterprise mobile applications. It also describes the deployment and management of in-house and commercially available mobile apps used in business settings on both company-provided and “bring your own” smartphones and tablet computers (BYOD).

Mobile application management has also been defined as “the strategy and process around developing/procuring, securing, deploying, accessing, configuring, updating and removing (business) applications from mobile devices used by the employees.” Enterprise MAM has been driven by the widespread adoption and use of mobile devices in business settings. IDC reported that smartphone use in the workplace will double between 2009 and 2014 and it continuously grows in future. The BYOD (“Bring Your Own Device”) phenomenon is a factor behind mobile application management, with personal PC, smartphone and tablet use in business settings (vs. business-owned devices) rising today in IT enterprise. When an employee brings a personal device into an enterprise setting, MAM enables the corporate IT staff to provision the device, download appropriate applications, control access to back-end data, and “wipe” the device if it is lost, or when its owner no longer works with the company.

Applications types which fall under MAM:

Corporate data access and decision support
CRM and sales force automation
Disaster recovery
Factory automation/manufacturing
GPS and location-based services
Human resources (timesheets, expense reports, etc.)
Inventory management
Order entry and tracking
Retail and point of sale
Training tools, including video

MAM Services:

App delivery
App security
App updating
User authentication
User authorization
Version checking
Push services
Reporting and tracking

Better Mobile Security Solutions:

Better MDM provides easy-to-use, cloud based enterprise grade technology that allows IT to adapt mobile architecture for their business in fast and secure way. Better MDM enables IT in enterprises, healthcare organizations and government agencies to secure the two end-points, Device and Applications.

Mobile App protection and Security Wrapping:

App Wrapping- Better MDM’s Mobile App Protection and Security Wrapping solution protects your apps through the use of fine-grained usage and security policies with no source code/SDK/API or changes to the apps required. For hyper-sensitive apps, multiple layers of protection can be used; strong authentication, data-at-rest encryption, VPN to protect data-in-transit and geo-fencing to restrict access based on geographic location. In addition, Better MDM’s Data Protection wrapper protects data from being copied and pasted to unauthorized applications. Better MDM provides both native and HTML 5 based app catalog apps.

Enterprise App Store- Enterprise App Store enables IT easily manage App deployment by selecting which applications they wish to make available to employees. With Better MDM’s solution, you can distribute, update and ensure the security of all enterprise applications.

App Security Analyzer- Better MDM’s App Security Analyzer identifies risks in in-house or third-party apps by scanning for malware, privacy breaches and corporate data exfiltration. We perform both static analysis of the app as well as running it instrumented emulation to identify and compute the risk score.

Security Policy Manager- Single Security Policy engine to apply and enforce corporate security policies on both mobile apps and devices.

MonoDevelop (IDE - Cross Platform): Design and develop compelling cutting edge client application software for the MacOS and iOS/Android platforms

2012-06-04T10:40:00.000-07:00

Hi Friends,

In this blog I am going to discuss about the IDE call "MonoDevelop" which helps the developer to develop and build the mobile client applications (i.e., iOS and Android) and services (i.e., SAAS) by little understanding of C# and ASP.Net. into the cross-platform environment.

Mono is a free and open source project led by Xamarin (formerly by Novell and originally by Ximian) to create an Ecma standard compliant .NET Framework-compatible set of tools including, among others, a C# compiler and a Common Language Runtime (CLR).

The stated purpose of Mono is not only to be able to run Microsoft .NET applications cross-platform, but also to bring better development tools to Linux developers. Mono can be run on many operating systems including Android, BSD, iOS, Linux, OS X, Windows, Solaris, Unix, and some for game consoles such as PlayStation 3, Wii, and Xbox 360.

Mono is having many innovative projects in their accounts, but few of them are so famous in respects of mobile application development and services environment. Several projects extend Mono and allow developers to use it in their development environment. These projects include:

1) Cross-Platform:

MonoCross, a cross-platform Model–view–controller design pattern where the Model and Controller are shared across platforms and the Views are unique for each platform for an optimized User Interface.
MonoDevelop an open source and cross platform Integrated Development Environment that supports building applications for ASP.NET, Gtk#, Meego, MonoTouch, iOS/iPad/iPod, Android and Silverlight/Moonlight.
Moonlight, an implementation of Silverlight that uses Mono. etc.

2) MacOS:

Cocoa# – wrappers around the native OS X toolkit (Cocoa) (deprecated).
Monobjc – a set of bindings for OS X programming.
MonoMac – newer bindings for OS X programming, based on the MonoTouch API design.

3) Mobile Platforms:

MonoDroid Mono for the Android operating system. With bindings for the Android APIs.
MonoTouch Mono for the iPhone, iPad and iPod Touches. With bindings to the iOS APIs.

MonoDevelop:

An open source IDE targeting both Mono and Microsoft .NET framework platforms. MonoDevelop is a free GNOME integrated development environment primarily designed for C# and other .NET languages such as Nemerle, Boo, and Java (via IKVM.NET), although it also supports languages such as C, C++, Python, Java, and Vala. MonoDevelop was originally a port of SharpDevelop to Gtk#, but it has since evolved to meet the needs of Mono developers. The IDE includes class management, built-in help, code completion, Stetic (a GUI designer), project support, and an integrated debugger. It supports MEAP concepts as well.

The MonoDoc browser provides access to API documentation and code samples. The documentation browser uses wiki-style content management, allowing developers to edit and improve the documentation.

1) OS Support:

OSX - MonoDevelop
Windows - Visual Studio 2010 Pro or above and MonoDevelop

2) Cross-platform:

Create amazing Android/iOS (iPhone/iPad) apps with C# and .NET

Cross Platform

Easily share code between iOS, Android and Windows Phone 7.
Native Code

High-performance compiled code with full access to all the native APIs, including UIKit.
C# & .NET

Everything you love about .NET, including LINQ, Delegates and Events.

MonoTouch and Mono for Android:

MonoTouch and Mono for Android, both developed by Xamarin, are implementations of Mono for iPhone and Android-based smart-phones. Contrary to Mono itself, they are released under a commercial license only.

MonoTouch - for iOS:

MonoTouch allows developers to create C# and .NET based applications that run on the iPhone. It is based on the Mono framework and developed in conjunction with Novell. Unlike Mono applications MonoTouch "Apps" are compiled down to machine code targeted specifically at the Apple iPhone. This is necessary because the iPhone kernel prevents just-in-time compilers from executing on the device.

The MonoTouch stack is made up of:

Compilers

C# from the Mono Project
Third party compilers like RemObject's Oxygene can target MonoTouch also

Core .NET libraries
Development SDK:

Linker – used to bundle only the code used in the final application
mtouch – the Native compiler and tool used to deploy to the target device
Interface Builder and storyboard integration tools

Libraries that bind the native CocoaTouch APIs
MonoDevelop IDE

MonoDevelop is used as the primary IDE however additional links to Xcode and the iOS simulator have been written.

From April to early September 2010, the future of MonoTouch was put in doubt as Apple introduced new terms for iPhone developers that apparently prohibits them from developing in languages other than C, C++ and Objective-C, and the use of a middle layer between the iOS platform and iPhone applications. This made the future of MonoTouch, and other technologies such as UNITY, uncertain.Then, in September 2010, Apple rescinded this restriction, stating that they were relaxing the language restrictions that they had put in place earlier that year.

Why you'll love MonoTouch.

MonoTouch makes creating iPhone and iPad apps easier than ever before.

Cross platform development

Thinking about supporting Android in the future? In many cases, you can reuse most of your existing code when porting from iOS to Mono for Android.
Discover iOS as you type

Explore APIs as you type with code autocompletion. Just like Visual Studio.
iOS 5 ready

MonoTouch has all of the latest iOS 5 features bound and ready to use in the apps you create.
Reuse existing code

Import existing .NET libraries and use them in your MonoTouch apps. Easily bind existing C and Objective-C libraries as well.
XCode 4 and UI designer integration

Use XCode 4 and the powerful UI Designer to create interfaces and Storyboards that automatically sync with your MonoTouch project.
Up to date

MonoTouch is frequently updated and ready with all of the new iOS features - typically on the same day they are released!

Mono for Android:

Mono for Android, developed by Xamarin, is a proprietary implementation of Mono for Android-based smart-phones. It was first released on April 6, 2011. Mono for Android was developed to allow developers to more easily write cross-platform applications that will run on all mobile platforms.

In reference to the lawsuit filed by Oracle against Google over Android's use of Java, that "The type of action Oracle is taking against Google over Java is not going to happen. If a .NET port to Android was through Mono it would fall under the Microsoft Community Promise Agreement." Recently Google has won against the Oracle filed lawsuit in favor for them about the same.

The Mono for Android stack consists of the following components:

Mono runtime
Libraries:

Core .NET class libraries
Libraries that bind the native Android/Java APIs

SDK tools to package, deploy and debug
Visual Studio 2010 integration to remotely debug and deploy.

Why you'll love Mono for Android.

Mono for Android makes creating Android apps easier than ever before.

Cross platform development

Thinking about supporting iPhone or iPad in the future? In many cases, you can reuse most of your existing code when porting from Android to iOS.
Discover as you type

Explore APIs as you type with code autocompletion.
Full featured

Mono for Android has all of the latest Android features bound and ready to use in the apps you create.
Reuse existing code

Import existing .NET libraries and use them in your Mono for Android apps.
The Full Power of Visual Studio

Continue using Visual Studio or use our full featured MonoDevelop IDE to create apps using Mono for Android.
Easy To Install

Installing the Android SDK can be time consuming, but Mono for Android simplifies the process by installing everything you need with just a few clicks.

Hadoop Overview - Big data Control though Mobile

2012-05-22T02:19:00.000-07:00

Hi Friends,

Here I am sharing about the overview of "Hadoop" framework and related projects which is necessary to improve the performance and maintains scalability for Big data. In future, I'll be adding how this stuffs are more useful for the Mobile end-client and computing in detail. Also, I have exposed here how Android smartphone does play with Hadoop concepts in practical.

Apache Hadoop is a software framework that supports data-intensive distributed applications under a free license. It enables applications to work with thousands of computational independent computers and petabytes of data. Hadoop was derived from Google's MapReduce and Google File System (GFS) papers.

Hadoop is a top-level Apache project being built and used by a global community of contributors, written in the Java programming language. Yahoo! has been the largest contributor to the project, and uses Hadoop extensively across its businesses.

"Hadoop is a framework for running applications on large clusters of commodity hardware. The Hadoop framework transparently provides applications both reliability and data motion. Hadoop implements a computational paradigm named map/reduce, where the application is divided into many small fragments of work, each of which may be executed or re-executed on any node in the cluster. In addition, it provides a distributed file system that stores data on the compute nodes, providing very high aggregate bandwidth across the cluster. Both map/reduce and the distributed file system are designed so that node failures are automatically handled by the framework."

Architecture

Hadoop consists of the Hadoop Common, which provides access to the filesystems supported by Hadoop. The Hadoop Common package contains the necessary JAR files and scripts needed to start Hadoop. The package also provides source code, documentation, and a contribution section which includes projects from the Hadoop Community.

For effective scheduling of work, every Hadoop-compatible filesystem should provide location awareness: the name of the rack (more precisely, of the network switch) where a worker node is. Hadoop applications can use this information to run work on the node where the data is, and, failing that, on the same rack/switch, so reducing backbone traffic. The Hadoop Distributed File System (HDFS) uses this when replicating data, to try to keep different copies of the data on different racks. The goal is to reduce the impact of a rack power outage or switch failure so that even if these events occur, the data may still be readable.

A small Hadoop cluster will include a single master and multiple worker nodes. The master node consists of a JobTracker, TaskTracker, NameNode, and DataNode. A slave or worker node acts as both a DataNode and TaskTracker, though it is possible to have data-only worker nodes, and compute-only worker nodes; these are normally only used in non-standard applications. Hadoop requires JRE 1.6 or higher.
In a larger cluster, the HDFS is managed through a dedicated NameNode server to host the filesystem index, and a secondary NameNode that can generate snapshots of the namenode's memory structures, thus preventing filesystem corruption and reducing loss of data. Similarly, a standalone JobTracker server can manage job scheduling. In clusters where the Hadoop MapReduce engine is deployed against an alternate filesystem, the NameNode, secondary NameNode and DataNode architecture of HDFS is replaced by the filesystem-specific equivalent.

Meet Hadoop:

We live in the data age. It’s not easy to measure the total volume of data stored elec- tronically, but an IDC estimate put the size of the “digital universe” at 0.18 zettabytes in 2006, and is forecasting a tenfold growth by 2011 to 1.8 zettabytes. A zettabyte is 1021 bytes, or equivalently one thousand exabytes, one million petabytes, or one billion terabytes. That’s roughly the same order of magnitude as one disk drive for every person in the world.
This flood of data is coming from many sources. Consider the following:

The New York Stock Exchange generates about one terabyte of new trade data per day.
Facebook hosts approximately 10 billion photos, taking up one petabyte of storage.
Ancestry.com, the genealogy site, stores around 2.5 petabytes of data.
The Internet Archive stores around 2 petabytes of data, and is growing at a rate of 20 terabytes per month.
The Large Hadron Collider near Geneva, Switzerland, will produce about 15 petabytes of data per year.etc,

So there’s a lot of data out there. But you are probably wondering how it affects you. Most of the data is locked up in the largest web properties (like search engines). The good news is that Big Data is here. The bad news is that we are struggling to store and analyze it.

Although Hadoop is best known for MapReduce and its distributed filesystem (HDFS, renamed from NDFS), the other subprojects provide complementary services, or build on the core to add higher-level abstractions. The subprojects, and where they sit in the technology stack, and described briefly here:

Core-A set of components and interfaces for distributed filesystems and general I/O (serialization, Java RPC, persistent data structures).

Avro- A data serialization system for efficient, cross-language RPC, and persistent data storage. (At the time of this writing, Avro had been created only as a new subproject, and no other Hadoop subprojects were using it yet.)

MapReduce- Hadoop's fundamental data filtering algorithm.A distributed data processing model and execution environment that runs on large clusters of commodity machines.

HDFS- A distributed filesystem that runs on large clusters of commodity machines.

Pig- A data flow language and execution environment for exploring very large datasets. Pig runs on HDFS and MapReduce clusters.

HBase- A distributed, column-oriented database. HBase uses HDFS for its underlying storage, and supports both batch-style computations using MapReduce and point queries (random reads).

ZooKeeper- A distributed, highly available coordination service. ZooKeeper provides primitives such as distributed locks that can be used for building distributed applications.

Hive- A distributed data warehouse. Hive manages data stored in HDFS and provides a query language based on SQL (and which is translated by the runtime engine to MapReduce jobs) for querying the data.

Chukwa- A distributed data collection and analysis system. Chukwa runs collectors that store data in HDFS, and it uses MapReduce to produce reports. (At the time of this writing, Chukwa had only recently graduated from a “contrib” module in Core to its own subproject.)

Some additional important Projects:

Nutch - an effort to build an open source search engine based on Lucene and Hadoop. Also created by Doug Cutting.
Datameer Analytics Solution (DAS) – data source integration, storage, analytics engine and visualization
Hypertable - HBase alternative
Apache Mahout - Machine Learning algorithms implemented on Hadoop
Apache Cassandra - A column-oriented database that supports access from Hadoop
HPCC - LexisNexis Risk Solutions High Performance Computing Cluster
Sector/Sphere - Open source distributed storage and processing

JobTracker and TaskTracker: the MapReduce engine

The file systems comes the MapReduce engine, which consists of one JobTracker, to which client applications submit MapReduce jobs. The JobTracker pushes work out to available TaskTracker nodes in the cluster, striving to keep the work as close to the data as possible. With a rack-aware filesystem, the JobTracker knows which node contains the data, and which other machines are nearby. If the work cannot be hosted on the actual node where the data resides, priority is given to nodes in the same rack. This reduces network traffic on the main backbone network. If a TaskTracker fails or times out, that part of the job is rescheduled.

The Job Tracker and TaskTracker status and information is exposed by Jetty and can be viewed from a web browser.

*If the JobTracker failed on Hadoop 0.20 or earlier, all ongoing work was lost. Hadoop version 0.21 added some checkpointing to this process; the JobTracker records what it is up to in the filesystem. When a JobTracker starts up, it looks for any such data, so that it can restart work from where it left off. In earlier versions of Hadoop, all active work was lost when a JobTracker restarted.

Known limitations of this approach are:

The allocation of work to TaskTrackers is very simple. Every TaskTracker has a number of available slots (such as "4 slots"). Every active map or reduce task takes up one slot. The Job Tracker allocates work to the tracker nearest to the data with an available slot. There is no consideration of the current system load of the allocated machine, and hence its actual availability.
If one TaskTracker is very slow, it can delay the entire MapReduce job - especially towards the end of a job, where everything can end up waiting for the slowest task. With speculative-execution enabled, however, a single task can be executed on multiple slave nodes.

Scheduling

By default Hadoop uses FIFO, and optional 5 scheduling priorities to schedule jobs from a work queue.In version 0.19 the job scheduler was refactored out of the JobTracker, and added the ability to use an alternate scheduler (such as the Fair scheduler or the Capacity scheduler).

Fair scheduler

The fair scheduler was developed by Facebook. The goal of the fair scheduler is to provide fast response times for small jobs and QoS for production jobs. The fair scheduler has three basic concepts.

Jobs are grouped into Pools.
Each pool is assigned a guaranteed minimum share.
Excess capacity is split between jobs.

By default jobs that are uncategorized go into a default pool. Pools have to specify the minimum number of map slots, reduce slots, and a limit on the number of running jobs.

Capacity scheduler

The capacity scheduler was developed by Yahoo. The capacity scheduler supports several features which are similar to the fair scheduler.

Jobs are submitted into queues.
Queues are allocated a fraction of the total resource capacity.
Free resources are allocated to queues beyond their total capacity.
Within a queue a job with a high level of priority will have access to the queue's resources.

There is no preemption once a job is running.

Hyrax:

The goal of Hyrax is to develop a mobile cloud infrastructure that enables smartphone applications with distributed data and computation. Hyrax allows applications to conveniently use data and execute computing jobs on smartphone networks and heterogeneous networks of phones and servers. Research has been focused primarily on implementation and evaluation of mobile cloud computing infrastructure based on MapReduce. As we read earlier, there are four types of processes in Hadoop instances: NameNode, JobTracker, DataNode, and TaskTracker. There is one NameNode and one JobTracker in a Hadoop cluster. NameNode schedules jobs and coordinates sub-tasks among TaskTrackers. A DataNode instance and a TaskTracker instance both run on each worker machine. DataNode stores and provides access to data blocks, and TaskTracker executes tasks assigned to it by JobTracker. Clients access files by first requesting block locations from NameNode and then requesting blocks directly from these locations.

Hyrax ports Hadoop to the Android platform. In Hyrax, each machine runs one instance of NameNode and one instance of JobTracker. DataNode and TaskTracker are run on each phone in separate Android service processes within the same application. Android applications may consist of multiple processes, some of which run as background services. Since DataNode and TaskTracker are run as Android services, they can run in the background of other applications.
A thread is also spawned to record information about the systemload—including power level and CPU, memory, network, and disk I/O statistics—into the local file system. Within the application, a server is run to allow external scripts to control data uploading, kill the program, and check the program status. Below Fig. illustrates the data interactions between all the software components on each phone.

Hyrax is implemented in a real testbed consisting of a cluster of 10 Android G1 (HTC Dream) phones and 5 HTC Magic phones, each running Android 1.5 Cupcake. Since Android does not support peer-to-peer networking yet, the phones communicate with each other on an isolated 802.11g network via a Linksys WRT54G wireless router with no firmware modifications. NameNode and JobTracker processes are run on a desktop machine connected behind this router via Ethernet. The phones are connected via USB to a controller that executes experiment scripts. These scripts are used to install Hyrax, initialize the cluster, run benchmarks, and collect and post process data.

To determine the advantages and drawbacks of Hyrax, an application was developed on it. The Hyrax multimedia search and sharing application, HyraxTube, allows users to browse videos and images stored on a network of phones and search by time, location, and quality. Quality ratings based on sensor data are generated by periodically executing a MapReduce job. Requests are serviced by reading results generated by the MapReduce job from HDFS. The client interface is implemented as a web application so that it can be used on mobile devices and desktop machines.

Sl. No.	Types of Database	Overview	Point to remember for Solutions Architect	Use Case
1	RDS	Managed PostgreSQL / MySQL / MariaDB / Oracle / SQL Server Must Provision an EC2 instance & EBS Volume type and size Support for Read Replicas and Multi-AZ Security through IAM, Security, Groups, KMS, SSL in transit Backup / Snapshot / Point in time restore feature Managed and scheduled maintenance Monitoring through Cloudwatch	Operations: small downtime when failover happens, when maintenance happens, scaling in read replicas / ec2 instance / restore EBS implies manual intervention, application changes Security: AWS responsible for OS security, we are responsible for setting up KMS, security groups, IAM policies, authoring users in DB, using SSL Reliability: Multi AZ feature, failover in case of failures Performance: depends on EC2 instance type, EBS volume type, ability to add Read Replicas. Doesn’t auto-scale Cost: Pay per hour based on provisional EC2 and EBS.	Store relational datasets (RDBMS/OLTP), perform SQL queries, transactional inserts/update/delete is available.
2	Aurora	Compatible API for PostgreSQL / MySQL Data is held in 6 replicas, across 3 AZ – lot of durability Auto healing capability Multi AZ, Auto scaling Read Replicas Read Replicas can be Global Aurora database can be Global for DR or latency purposes Auto scaling of storage from 10GB to 64TB Define EC2 instance type for aurora instances Same security / monitoring / maintenance features as RDS “Aurora Serverless” option	Operations: less operations, auto scaling storage Security: AWS responsible for OS security, we are responsible for setting up KMS, security groups, IAM policies, authoring users in DB using SSL Reliability: multi AZ, highly available, possibly more than RDS, Aurora Serverless option Performance: 5x performance (according to AWS) due to architectural optimization. Up to 15 Read Replicas (only 5 for RDS) Cost: Pay per hour based on EC2 and storage usage. Possibly lower costs compared to Enterprise grade databases such as Oracle	Same as RDS, but with less maintenance / more flexibility / more performance. *For enterprise grade applications, Aurora is the best option to choose.
3	ElastiCache	Managed Redis / Memcached (similar offering as RDS, but for caches) In-memory data store, sub-millisecond latency Must proviso an EC2 instance type Support for Clustering (Redis) and Multi AZ, Read Replicas (sharding) Security through IAM, Security groups, KMS, Redis Auth Backup / Snapshot / Point in time restore feature Managed scheduled maintenance Monitoring through CloudWatch	Operations: same as RDS Security: AWS responsible for OS security, we are responsible for setting up KMS, security groups, IAM policies, users (Redis Auth), using SSL Reliability: multi AZ, Clustering, Sharding Performance: Sub-millisecond performance, in memory, read replicas for sharding, very popular cache option Cost: Pay per hour based on EC2 and storage usage.	Key/Value store, Frequent reads, less writes, cache results for DB queries, store session data for websites, cannot use SQL.
4	DynamoDB	DynamoDB is a pure cloud native technology, it’s a serverless AWS proprietary technology, managed NoSQL database Serverless, provisioned capacity, auto scaling, on demand capacity (Nov 2018) – scales based on your load Can replace ElastiCache as a key/value store (storing session data for example) Highly available, multi AZ by default, Reads and Writes are decoupled, DAX for read cache Reads can be eventually consistent or strongly consistent Security, authentication and authorization is done through IAM DynamoDB streams to integrate with AWS Lambda Backup / Restore feature, Global Table feature Monitoring through CloudWatch Can only query on primary key, sort key, or indexes	Operations: no operations needed, auto scaling capability, serverless Security: full security through IAM policies, KMS encryption, SSL in flight Reliability: multi AZ, Backups Performance: single digit millisecond performance, DAX for caching reads, performance doesn’t degrade if your application scales Cost: Pay per provisioned capacity and storage usage (no need to guess in advance any capacity – can use auto scaling).	Serverless applications development (small documents 100s KB), distributed serverless cache, doesn’t have SQL query language available, has transactions capability from NOV 2018.
5	S3	S3 is a database, it’s not a conventional database S3 is a …key / value store for objects Great for big objects, not so great for small objects S3 doesn’t replace RDS / DynamoDB Serverless, scales infinitely, max object size is 5TB Eventually consistency for overwrites and deletes Tiers: S3 Standard, S3 IA (Infrequent Access), S3 One Zone IA, Glacier for backups Features: versioning, encryption, CRR (Cross Region Replication), etc… Security: IAM, Bucket policies, ACL (Access Control List) Encryption: SSE-S3, SSE-KMS, SSE-C, client side encryption, SSL in transit	Operations: no operations needed Security: IAM, Bucket Policies, ACL, Encryption (Server/Client), SSL Reliability: 99.999999999% durability / 99.99% availability, multi AZ, CRR Performance: scales to thousands of reads / writes per second, transfer acceleration / multi-part for big files Cost: pay per storage usage, network cost, request number	Static files, key value store for big files, website hosting.
6	Athena	It is not a database it terms but it holds the data but it does provide a query engine on top of S3. Fully serverless database with SQL capabilities Used to query data in S3 Pay per query Output results back to S3 Secured through IAM	Operations: no operations needed, serverless Security: IAM + S3 security Reliability: managed service, used Presto* engine, highly available Performance: queries scale based on data size Cost: pay per query / per TB of data scanned, serverless	One time SQL queries, serverless queries on S3, log analytics.
7	Redshift	Redshift is based on PostgreSQL, but it’s not used for OLTP It’s OLAP – online analytical processing (analytics and data warehousing) 10x better performance than other data warehouses, scale to PBs of data Columnar storage of data (instead of row based) Massively Parallel Query Execution (MPP), highly available Pay as you go based on the instance provisioned Has a SQL interface for performing the queries BI tools such as AWS Quicksight or Tableau integrate with it Data is loaded from S3, DynamoDB, DMS, other DBs… From 1 node to 128 nodes, up to 160 GB of space per node Leader node: for query planning, results aggregation Compute node: for performing the queries, send results to leader Redshift Spectrum: perform queries directly against S3 (no need to load) Backup & Restore, Security VPC / IAM / KMS, Monitoring Redshift Enhanced VPC Routing: COPY / UNLOAD goes through VPC	Operations: similar to RDS Security: IAM, VPC, KMS, SSL (Similar to RDS) Reliability: highly available, auto healing features Performance: 10x performance vs other data warehousing, compression Cost: pay per node provisioned, 1/10^th of the cost vs other warehouses	Remember: Redshift = Analytics / BI/ Data Warehouse
8	Neptune	Fully managed graph database When do we use Graphs? High relationship data Social Networking: Users friends with Users, replied to comment on post of user and likes other comments Knowledge graphs (Wikipedia) Highly available across 3 AZ, with up to 15 read replicas Point-in-time recovery, continuous backup to Amazon S3 Support for KMS encryption at rest + HTTPS	Operations: similar to RDS Security: IAM, VPC, KMS, SSL (similar to RDS) + IAM Authentication Reliability: Multi-AZ, clustering Performance: best suited for graphs, clustering to improve performance Cost: pay per node provisioned (similar to RDS)	Remember: Neptune = Graphs
9	ElasticSearch	With ElasticSearch, you can search any field, even partially matches It’s common to use ElasticSearch as a complement to another database ElasticSearch also has some usage for Big Data applications You can provision a cluster of instances Built-in integrations: Amazon Kinesis Data Firehose, AWS IoT, and Amazon CloudWatch Logs for data ingestion Security through Cognito & IAM, KMS encryption, SSL & VPC Comes with Kibana (visualization) & Logstash (log ingestion) – ELK stack	Operations: similar to RDS Security: Cognito, IAM, VPC, KMS, SSL Reliability: Multi-AZ, clustering Performance: best on ElasticSearch project (open source), petabyte scale Cost: pay per node provisioned (similar to RDS)	Remember: ElasticSearch = Search / Indexing