Education Everywhere

Amazon Simple Storage Service

2018-10-07T21:24:00.003-07:00

Amazon Simple Storage Service

Amazon Simple Storage Service is storage for the Internet. It is designed to make web-scale computing easier for developers.
Amazon S3 has a simple web services interface that you can use to store and retrieve any amount of data, at any time, from anywhere on the web. It gives any developer access to the same highly scalable, reliable, fast, inexpensive data storage infrastructure that Amazon uses to run its own global network of web sites. The service aims to maximize benefits of scale and to pass those benefits on to developers.

Installation & Setup

Prerequisites

To use the AWS SDK for Java, you must have:

AWS Free tier account click here and register .This enables you to gain free, hands-on experience with the AWS platform, products, and services.
a suitable Java Development Environment.
An AWS account and access keys. For instructions, see Sign Up for AWS and Create an IAM User.
AWS credentials (access keys) set in your environment or using the shared (by the AWS CLI and other SDKs) credentials file. For more information, see Set up AWS Credentials and Region for Development.

Steps

A> Install Maven

Visit Maven official website, download the Maven zip file,
Add MAVEN_HOME variables in the Windows environment, and point it to your Maven folder.
Update PATH variable, append Maven bin folder – %M2_HOME%\bin, so that you can run the Maven’s command everywhere.
Done, to verify it, run mvn –version in the command prompt.

C:\Users\>mvn -version

B> AWS Toolkit

Prerequisites

An Amazon Web Services account– To obtain an AWS account, go to the AWS home page and click Sign Up Now. Signing up will enable you to use all of the services offered by AWS.
A supported operating system– The AWS Toolkit for Eclipse is supported on Windows, Linux, macOS, or Unix.
Java 1.8 or later
Eclipse IDE for Java Developers 4.2 or later– We attempt to keep the AWS Toolkit for Eclipse current with the default version available on the Eclipse download page

Steps

1. Open Eclipse and go to Help > Eclipse marketplace and search for AWS Toolkit

2. After Successfully Installation, You will see new Icon for AWS Toolkit as shown below in Eclipse menu bar.

3. Create AWS Java Project and select appropriate checkbox’s for Amazon Services which are needed in project.

4. Go to the AWSToolkit > Preferences and Fill up Credentials details.

Getting Started

AWS IAM (Identity and Access Management ) :-

It enables you to manage access to AWS services and resources securely. Using IAM, you can create and manage AWS users and groups, and use permissions to allow and deny their access to AWS resources.
AWS has a list of best practices to help IT professionals and developers manage access to AWS resources.
Users – Create individual users.
Groups – Manage permissions with groups.
Permissions – Grant least privilege.
Auditing – Turn on AWS CloudTrail.
Password – Configure a strong password policy.
MFA – Enable MFA for privileged users.
Roles – Use IAM roles for Amazon EC2 instances.
Sharing – Use IAM roles to share access.
Rotate – Rotate security credentials regularly.
Conditions – Restrict privileged access further with conditions.
Root – Reduce or remove use of root.

Steps for Create IAM User

1. Go to AWS Console and click on IAM

2. Create group and assign AmazonS3FullAccess Policy

3. Create user and assign to group and click

Sample Code for Amazon S3

package com.amazonaws.samples; /* * Copyright 2010-2017 Amazon.com, Inc. or its affiliates. All Rights Reserved. */ import java.io.BufferedReader; import java.io.File; import java.io.FileOutputStream; import java.io.IOException; import java.io.InputStream; import java.io.InputStreamReader; import java.io.OutputStreamWriter; import java.io.Writer; import java.util.UUID;
import com.amazonaws.AmazonClientException;
import com.amazonaws.AmazonServiceException;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.regions.Region;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.Bucket;
import com.amazonaws.services.s3.model.GetObjectRequest;
import com.amazonaws.services.s3.model.ListObjectsRequest;
import com.amazonaws.services.s3.model.ObjectListing;
import com.amazonaws.services.s3.model.PutObjectRequest;
import com.amazonaws.services.s3.model.S3Object;
import com.amazonaws.services.s3.model.S3ObjectSummary;
public class S3Sample {
public static void main(String[] args) throws IOException {
AWSCredentials credentials = null;
try {
credentials = new ProfileCredentialsProvider("default").getCredentials();
} catch (Exception e) {
throw new AmazonClientException(
"Cannot load the credentials from the credential profiles file. " +
"Please make sure that your credentials file is at the correct " +
"location (C:\\Users\\Shrikant_Jagtap\\.aws\\credentials), and is in valid format.",
e);
}
AmazonS3 s3 = AmazonS3ClientBuilder.standard()
.withCredentials(new AWSStaticCredentialsProvider(credentials))
.withRegion("us-west-2")
.build();
String bucketName = "my-first-s3-bucket-" + UUID.randomUUID();
String key = "MyObjectKey";
System.out.println("===========================================");
System.out.println("Getting Started with Amazon S3");
System.out.println("===========================================\n");
try {
System.out.println("Creating bucket " + bucketName + "\n");
s3.createBucket(bucketName);
System.out.println("Listing buckets");
for (Bucket bucket : s3.listBuckets()) {
System.out.println(" - " + bucket.getName());
}
System.out.println();
System.out.println("Uploading a new object to S3 from a file\n");
s3.putObject(new PutObjectRequest(bucketName, key, createSampleFile()));
System.out.println("Downloading an object");
S3Object object = s3.getObject(new GetObjectRequest(bucketName, key));
System.out.println("Content-Type: " + object.getObjectMetadata().getContentType());
displayTextInputStream(object.getObjectContent());
System.out.println("Listing objects");
ObjectListing objectListing = s3.listObjects(new ListObjectsRequest()
.withBucketName(bucketName)
.withPrefix("My"));
for (S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) {
System.out.println(" - " + objectSummary.getKey() + " " +
"(size = " + objectSummary.getSize() + ")");
}
System.out.println();
System.out.println("Deleting an object\n");
s3.deleteObject(bucketName, key);
System.out.println("Deleting bucket " + bucketName + "\n");
s3.deleteBucket(bucketName);
} catch (AmazonServiceException ase) {
System.out.println("Caught an AmazonServiceException, which means your request made it "
+ "to Amazon S3, but was rejected with an error response for some reason.");
System.out.println("Error Message: " + ase.getMessage());
System.out.println("HTTP Status Code: " + ase.getStatusCode());
System.out.println("AWS Error Code: " + ase.getErrorCode());
System.out.println("Error Type: " + ase.getErrorType());
System.out.println("Request ID: " + ase.getRequestId());
} catch (AmazonClientException ace) {
System.out.println("Caught an AmazonClientException, which means the client encountered "
+ "a serious internal problem while trying to communicate with S3, "
+ "such as not being able to access the network.");
System.out.println("Error Message: " + ace.getMessage());
}
}
private static File createSampleFile() throws IOException {
File file = File.createTempFile("aws-java-sdk-", ".txt");
file.deleteOnExit();
Writer writer = new OutputStreamWriter(new FileOutputStream(file));
writer.write("abcdefghijklmnopqrstuvwxyz\n");
writer.write("01234567890112345678901234\n");
writer.write("!@#$%^&*()-=[]{};':',.<>/?\n");
writer.write("01234567890112345678901234\n");
writer.write("abcdefghijklmnopqrstuvwxyz\n");
writer.close();
return file;
}
/**
* Displays the contents of the specified input stream as text.
*
* @param input
* The input stream to display as text.
*
* @throws IOException
*/
private static void displayTextInputStream(InputStream input) throws IOException {
BufferedReader reader = new BufferedReader(new InputStreamReader(input));
while (true) {
String line = reader.readLine();
if (line == null) break;
System.out.println(" " + line);
}
System.out.println();
}
}

CONSOLE OUPUT
===========================================
Getting Started with Amazon S3
===========================================
Creating bucket my-first-s3-bucket-00802f76-777d-41ea-b236-650286137f26
Listing buckets
- my-first-s3-bucket-00802f76-777d-41ea-b236-650286137f26
- myshribucket
- shribucket123
Uploading a new object to S3 from a file
Downloading an object
Content-Type: text/plain
abcdefghijklmnopqrstuvwxyz
01234567890112345678901234
!@#$%^&*()-=[]{};':',.<>/?
01234567890112345678901234
abcdefghijklmnopqrstuvwxyz
Listing objects
- MyObjectKey (size = 135)
Deleting an object
Deleting bucket my-first-s3-bucket-00802f76-777d-41ea-b236-650286137f26

AWS Command Line

The AWS Command Line Interface (CLI) is a unified tool to manage your AWS services. With just one tool to download and configure, you can control multiple AWS services from the command line and automate them through scripts.

Steps for Install AWS CLI

1. Download And Install for window by Click Here

2. Check once everything has been setup correctly

3. you can configure it using the following command

Note

Here: us-west-2) and the default output format (here: json). If you need different profiles, you can specify this additionally using the argument --profile:

aws configure --profile my-second-profile

A: Buckets Operations & Commands

High-level commands for the creation and deletion of buckets are provided by the aws-cli tool. Just provide the string s3as first argument to aws followed by a shortcut for “make bucket”, “remove bucket”:

Ø Create New Bucket.

Ø Delete buckets that are already empty. Removing buckets with contents can be done by providing the option --force

Ø To list the contents of a bucket

Ø List all available buckets:

B: Objects operations &commands

The command line interface supports high-level operations for uploading, moving and removing objects.

Ø Uploading a local file to s3

Ø Specify the storage class for the new remote object:

Ø new file can be removed by using the command rm

Ø move complete sets of files from S3 to the local machine

C: Synchronization operations &commands

Often it is useful to synchronize complete folder structures and their content either from the local machine to S3 or vice versa. Therefor the aws-cli tool comes with the handy option sync

Ø updates all files in S3 that have a different size and/or modification timestamp than the one in the local directory. As it does not remove objects from the S3,

Ø you must specify the --delete option to let the tool also remove files in S3 that are not present in your local copy

Ø When synchronizing your local copy with the remote files in S3 you can also specify the storage class and the access privilege’s

Refernce: https://aws.amazon.com/getting-started/

Introduction to Python

2018-10-03T00:26:00.000-07:00

Introduction to Python

Python is a general-purpose interpreted, interactive, object-oriented, and high-level programming language. It was created by Guido van Rossum during 1985- 1990. Like Perl, Python source code is also available under the GNU General Public License (GPL).

Python Features

Interpreted− Python is processed at runtime by the interpreter. You do not need to compile your program before executing it. This is similar to PERL and PHP.
Object-Oriented− Python supports Object-Oriented style or technique of programming that encapsulates code within objects.
A broad standard library − Python's bulk of the library is very portable and cross-platform compatible on UNIX, Windows, and Macintosh.
Interactive Mode− Python has support for an interactive mode which allows interactive testing and debugging of snippets of code.
Portable− Python can run on a wide variety of hardware platforms and has the same interface on all platforms.
Databases− Python provides interfaces to all major commercial databases.
GUI Programming− Python supports GUI applications that can be created and ported to many system calls, libraries and windows systems, such as Windows MFC, Macintosh, and the X Window system of Unix.

Installation & Setup

Python is available on a wide variety of platforms including Linux and Mac OS X.
official website of Python https://www.python.org/
Documentation https://www.python.org/doc/

Unix and Linux :

Open a Web browser and go to https://www.python.org/downloads/.
Download and extract files.
Editing the Modules/Setup file if you want to customize some options.
run ./configure script and make install

Windows :

Open a Web browser and go to https://www.python.org/downloads/.
To use this installer python-XYZ.msi, the Windows system must support Microsoft Installer 2.0. Save the installer file to your local machine and then run it to find out if your machine supports MSI.
Run the downloaded file. This brings up the Python install wizard, which is really easy to use. Just accept the default settings, wait until the install is finished, and you are done.

Macintosh :

Recent Macs come with Python installed, but it may be several years out of date. See http://www.python.org/download/mac/ for instructions on

Editors and IDEs

General Editors

Eclipse + PyDev
Sublime Text
Atom
GNU Emacs
Vi / Vim
Visual Studio
Visual Studio Code

Recommended Python-Specific

PyCharm
Spyder
Thonny

Install Pycharm

The official website of PyCharm
https://www.jetbrains.com/pycharm/download/#section=windows
Here you will observe two versions of package for Windows as shown in the screenshot
Download the community package (executable file) and install onto your system

Python Introduction

2018-10-03T00:21:00.000-07:00

Introduction to Python

Python Features

Interpreted− Python is processed at runtime by the interpreter. You do not need to compile your program before executing it. This is similar to PERL and PHP.
Object-Oriented− Python supports Object-Oriented style or technique of programming that encapsulates code within objects.
A broad standard library − Python's bulk of the library is very portable and cross-platform compatible on UNIX, Windows, and Macintosh.
Interactive Mode− Python has support for an interactive mode which allows interactive testing and debugging of snippets of code.
Portable− Python can run on a wide variety of hardware platforms and has the same interface on all platforms.
Databases− Python provides interfaces to all major commercial databases.
GUI Programming− Python supports GUI applications that can be created and ported to many system calls, libraries and windows systems, such as Windows MFC, Macintosh, and the X Window system of Unix.

Installation & Setup

Python is available on a wide variety of platforms including Linux and Mac OS X.
official website of Python https://www.python.org/
Documentation https://www.python.org/doc/

Unix and Linux :

Open a Web browser and go to https://www.python.org/downloads/.
Download and extract files.
Editing the Modules/Setup file if you want to customize some options.
run ./configure script and make install

Windows :

Open a Web browser and go to https://www.python.org/downloads/.
To use this installer python-XYZ.msi, the Windows system must support Microsoft Installer 2.0. Save the installer file to your local machine and then run it to find out if your machine supports MSI.
Run the downloaded file. This brings up the Python install wizard, which is really easy to use. Just accept the default settings, wait until the install is finished, and you are done.

Macintosh :

Recent Macs come with Python installed, but it may be several years out of date. See http://www.python.org/download/mac/ for instructions on

Editors and IDEs

General Editors

Eclipse + PyDev
Sublime Text
Atom
GNU Emacs
Vi / Vim
Visual Studio
Visual Studio Code

Recommended Python-Specific

PyCharm
Spyder
Thonny