Database system, Information security, Math Online Exercises, Past Papers - Class 11 and Class 12,

Decision tree

2025-04-18T22:20:00.000-07:00

Decision tree

· A decision tree is a tree-based supervised learning method used to predict the output of a target variable.

· Supervised learning uses labeled data (data with known output variables) to make predictions with the help of regression and classification algorithms.

· Supervised learning algorithms act as a supervisor for training a model with a defined output variable.

· It learns from simple decision rules using the various data features.

· Decision trees in Python can be used to solve both classification and regression problems—they are frequently used in determining odds.

Useful Concepts

The decision tree algorithm breaks down a dataset into smaller subsets; while during the same time, an associated decision tree is incrementally developed. A decision tree consists of nodes (that test for the value of a certain attribute), edges/branch (that correspond to the outcome of a test and connect to the next node or leaf) & leaf nodes (the terminal nodes that predict the outcome) that makes it a complete structure.

An example of a decision tree can be explained using above binary tree. Let’s say you want to predict whether a person is fit given their information like age, eating habit, and physical activity, etc. The decision nodes here are questions like ‘What’s the age?’, ‘Does he exercise?’, and ‘Does he eat a lot of pizzas’? And the leaves, which are outcomes like either ‘fit’, or ‘unfit’. In this case this was a binary classification problem (a yes no type problem). There are two main types of Decision Trees:

1. Classification trees (Yes/No types)

A classification tree is an algorithm where the target variable is fixed or categorical. The algorithm is then used to identify the “class” within which a target variable would most likely fall.

An example of a classification-type problem would be determining who will or will not subscribe to a digital platform; or who will or will not graduate from high school.

These are examples of simple binary classifications where the categorical dependent variable can assume only one of two, mutually exclusive values. In other cases, you might have to predict among a number of different variables. For instance, you may have to predict which type of smartphone a consumer may decide to purchase.

In such cases, there are multiple values for the categorical dependent variable. Here’s what a classic classification tree looks like

1. Regression trees (Continuous data types)

A regression tree refers to an algorithm where the target variable is and the algorithm is used to predict it’s value. As an example of a regression type problem, you may want to predict the selling prices of a residential house, which is a continuous dependent variable.

This will depend on both continuous factors like square footage as well as categorical factors like the style of home, area in which the property is located and so on.

Important Terms Used in Decision Trees

1. Entropy: Entropy is the measure of uncertainty or randomness in a data set. Entropy handles how a decision tree splits the data.

It is calculated using the following formula:

2. Information Gain: The information gain measures the decrease in entropy after the data set is split.

It is calculated as follows:

IG( Y, X) = Entropy (Y) - Entropy ( Y | X)

3. Gini Index: The Gini Index is used to determine the correct variable for splitting nodes. It measures how often a randomly chosen variable would be incorrectly identified.

4. Root Node: The root node is always the top node of a decision tree. It represents the entire population or data sample, and it can be further divided into different sets.

5. Decision Node: Decision nodes are sub-nodes that can be split into different sub-nodes; they contain at least two branches.

6. Leaf Node: A leaf node in a decision tree carries the final results. These nodes, which are also known as terminal nodes, cannot be split any further.

How to avoid/counter Overfitting in Decision Trees?

The common problem with Decision trees, especially having a table full of columns, they fit a lot. Sometimes it looks like the tree memorized the training data set. If there is no limit set on a decision tree, it will give you 100% accuracy on the training data set because in the worse case it will end up making 1 leaf for each observation. Thus this affects the accuracy when predicting samples that are not part of the training set.

Here are two ways to remove overfitting:

1. Pruning Decision Trees.

2. Random Forest

Pruning Decision Trees:

The splitting process results in fully grown trees until the stopping criteria are reached. But, the fully grown tree is likely to overfit the data, leading to poor accuracy on unseen data.

In pruning, you trim off the branches of the tree, i.e., remove the decision nodes starting from the leaf node such that the overall accuracy is not disturbed. This is done by segregating the actual training set into two sets: training data set, D and validation data set, V. Prepare the decision tree using the segregated training data set, D. Then continue trimming the tree accordingly to optimize the accuracy of the validation data set, V.

In the above diagram, the ‘Age’ attribute in the left-hand side of the tree has been pruned as it has more importance on the right-hand side of the tree, hence removing overfitting.

Random Forest:

Random Forest is an example of ensemble learning, in which we combine multiple machine learning algorithms to obtain better predictive performance.

While implementing the decision tree we will go through the following two phases:

1. Building Phase

• Preprocess the dataset.

• Split the dataset from train and test using Python sklearn package.

• Train the classifier.

2. Operational Phase

• Make predictions.

• Calculate the accuracy.

ID3:

There are many algorithms out there which construct Decision Trees, but one of the best is called as ID3 Algorithm. ID3 Stands for Iterative Dichotomiser 3. Before discussing the ID3 algorithm, we’ll go through few definitions. Entropy Entropy, also called as Shannon Entropy is denoted by H(S) for a finite set S, is the measure of the amount of uncertainty or randomness in data.

Intuitively, it tells us about the predictability of a certain event. Example, consider a coin toss whose probability of heads is 0.5 and probability of tails is 0.5. Here the entropy is the highest possible, since there’s no way of determining what the outcome might be. Alternatively, consider a coin which has heads on both the sides, the entropy of such an event can be predicted perfectly since we know beforehand that it’ll always be heads. In other words, this event has no randomness hence it’s entropy is zero. In particular, lower values imply less uncertainty while higher values imply high uncertainty. Information Gain Information gain is also called as Kullback-Leibler divergence denoted by IG(S,A) for a set S is the effective change in entropy after deciding on a particular attribute A. It measures the relative change in entropy with respect to the independent variables

IG(S, A) = H (S )- H (S, A)

Alternatively,

IG(S, A) = H (S )- å P(x)* H (x)

i=0

where IG(S, A) is the information gain by applying feature A. H(S) is the Entropy of the entire set, while the second term calculates the Entropy after applying the feature A, where P(x) is the probability of event x. Let’s understand this with the help of an example Consider a piece of data collected over the course of 14 days where the features are Outlook, Temperature, Humidity, Wind and the outcome variable is whether Golf was played on the day. Now, our job is to build a predictive model which takes in above 4 parameters and predicts whether Golf will be played on the day. We’ll build a decision tree to do that using ID3 algorithm.

Day	Outlook	Temperature	Humidity	Wind	Play Golf
D1	Sunny	Hot	High	Weak	No
D2	Sunny	Hot	High	Strong	No
D3	Overcast	Hot	High	Weak	Yes
D4	Rain	Mild	High	Weak	Yes
D5	Rain	Cool	Normal	Weak	Yes
D6	Rain	Cool	Normal	Strong	No
D7	Overcast	Cool	Normal	Strong	Yes
D8	Sunny	Mild	High	Weak	No
D9	Sunny	Cool	Normal	Weak	Yes
D10	Rain	Mild	Normal	Weak	Yes
D11	Sunny	Mild	Normal	Strong	Yes
D12	Overcast	Mild	High	Strong	Yes
D13	Overcast	Hot	Normal	Weak	Yes
D14	Rain	Mild	High	Strong	No

Multiple Linear Regressions

2025-04-18T09:37:00.000-07:00

Multiple Linear Regressions:-

Multiple linear regression (MLR), also known simply as multiple regression, is a statistical technique that uses several explanatory variables to predict the outcome of a response variable. The goal of multiple linear regression is to model the linear relationship between the explanatory (independent) variables and response (dependent) variables. In essence, multiple regression is the extension of ordinary least-squares (OLS) regression because it involves more than one explanatory variable.

Multiple linear regression (MLR), also known simply as multiple regression, is a statistical technique that uses several explanatory variables to predict the outcome of a response variable.

Multiple regression is an extension of linear (OLS) regression that uses just one explanatory variable.

What Multiple Linear Regression Can Tell You

Simple linear regression is a function that allows an analyst or statistician to make predictions about one variable based on the information that is known about another variable. Linear regression can only be used when one has two continuous variables an independent variable and a dependent variable. The independent variable is the parameter that is used to calculate the dependent variable or outcome. A multiple regression model extends to several explanatory variables.

The multiple regression model is based on the following assumptions:

There is a linear relationship between the dependent variables and the independent variables The independent variables are not too highly correlated with each other yi observations are selected independently and randomly from the population.

code:

import numpy as np

import matplotlib.pyplot as plt

import pandas as pd

"""## Importing the dataset"""

## dataset = pd.read_csv(r'D:\Regression\50_Startups.csv')

dataset = pd.read_csv("C:/Users/SR Laptop/Desktop/multiple linear regression/50_Startups.csv")

X = dataset.iloc[:, :-1].values

y = dataset.iloc[:, -1].values

print(X)

"""## Encoding categorical data"""

from sklearn.compose import ColumnTransformer

from sklearn.preprocessing import OneHotEncoder

ct = ColumnTransformer(transformers=[('encoder', OneHotEncoder(), [3])],

remainder='passthrough')

X = np.array(ct.fit_transform(X))

print(X)

"""## Splitting the dataset into the Training set and Test set"""

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2,

random_state = 0)

"""## Training the Multiple Linear Regression model on the Training set"""

from sklearn.linear_model import LinearRegression

regressor = LinearRegression()

regressor.fit(X_train, y_train)

"""## Predicting the Test set results"""

y_pred = regressor.predict(X_test)

np.set_printoptions(precision=2)

print(np.concatenate((y_pred.reshape(len(y_pred),1),

y_test.reshape(len(y_test),1)),1))

"""## Predicting the Test set results in other way"""

y_pred = regressor.predict(X_test)

np.set_printoptions(precision=2)

# Print header

print("Predicted Profit\tActual Profit")

print("----------------------------------")

# Loop to print each pair

for pred, actual in zip(y_pred, y_test):

print(f"{pred:,.2f}\t\t{actual:,.2f}")

Output:

[[165349.2 136897.8 471784.1 'New York']
 [162597.7 151377.59 443898.53 'California']
 [153441.51 101145.55 407934.54 'Florida']
 [144372.41 118671.85 383199.62 'New York']
 [142107.34 91391.77 366168.42 'Florida']
 [131876.9 99814.71 362861.36 'New York']
 [134615.46 147198.87 127716.82 'California']
 [130298.13 145530.06 323876.68 'Florida']
 [120542.52 148718.95 311613.29 'New York']
 [123334.88 108679.17 304981.62 'California']
 [101913.08 110594.11 229160.95 'Florida']
 [100671.96 91790.61 249744.55 'California']
 [93863.75 127320.38 249839.44 'Florida']
 [91992.39 135495.07 252664.93 'California']
 [119943.24 156547.42 256512.92 'Florida']
 [114523.61 122616.84 261776.23 'New York']
 [78013.11 121597.55 264346.06 'California']
 [94657.16 145077.58 282574.31 'New York']
 [91749.16 114175.79 294919.57 'Florida']
 [86419.7 153514.11 0.0 'New York']
 [76253.86 113867.3 298664.47 'California']
 [78389.47 153773.43 299737.29 'New York']
 [73994.56 122782.75 303319.26 'Florida']
 [67532.53 105751.03 304768.73 'Florida']
 [77044.01 99281.34 140574.81 'New York']
 [64664.71 139553.16 137962.62 'California']
 [75328.87 144135.98 134050.07 'Florida']
 [72107.6 127864.55 353183.81 'New York']
 [66051.52 182645.56 118148.2 'Florida']
 [65605.48 153032.06 107138.38 'New York']
 [61994.48 115641.28 91131.24 'Florida']
 [61136.38 152701.92 88218.23 'New York']
 [63408.86 129219.61 46085.25 'California']
 [55493.95 103057.49 214634.81 'Florida']
 [46426.07 157693.92 210797.67 'California']
 [46014.02 85047.44 205517.64 'New York']
 [28663.76 127056.21 201126.82 'Florida']
 [44069.95 51283.14 197029.42 'California']
 [20229.59 65947.93 185265.1 'New York']
 [38558.51 82982.09 174999.3 'California']
 [28754.33 118546.05 172795.67 'California']
 [27892.92 84710.77 164470.71 'Florida']
 [23640.93 96189.63 148001.11 'California']
 [15505.73 127382.3 35534.17 'New York']
 [22177.74 154806.14 28334.72 'California']
 [1000.23 124153.04 1903.93 'New York']
 [1315.46 115816.21 297114.46 'Florida']
 [0.0 135426.92 0.0 'California']
 [542.05 51743.15 0.0 'New York']
 [0.0 116983.8 45173.06 'California']]
[[0.0 0.0 1.0 165349.2 136897.8 471784.1]
 [1.0 0.0 0.0 162597.7 151377.59 443898.53]
 [0.0 1.0 0.0 153441.51 101145.55 407934.54]
 [0.0 0.0 1.0 144372.41 118671.85 383199.62]
 [0.0 1.0 0.0 142107.34 91391.77 366168.42]
 [0.0 0.0 1.0 131876.9 99814.71 362861.36]
 [1.0 0.0 0.0 134615.46 147198.87 127716.82]
 [0.0 1.0 0.0 130298.13 145530.06 323876.68]
 [0.0 0.0 1.0 120542.52 148718.95 311613.29]
 [1.0 0.0 0.0 123334.88 108679.17 304981.62]
 [0.0 1.0 0.0 101913.08 110594.11 229160.95]
 [1.0 0.0 0.0 100671.96 91790.61 249744.55]
 [0.0 1.0 0.0 93863.75 127320.38 249839.44]
 [1.0 0.0 0.0 91992.39 135495.07 252664.93]
 [0.0 1.0 0.0 119943.24 156547.42 256512.92]
 [0.0 0.0 1.0 114523.61 122616.84 261776.23]
 [1.0 0.0 0.0 78013.11 121597.55 264346.06]
 [0.0 0.0 1.0 94657.16 145077.58 282574.31]
 [0.0 1.0 0.0 91749.16 114175.79 294919.57]
 [0.0 0.0 1.0 86419.7 153514.11 0.0]
 [1.0 0.0 0.0 76253.86 113867.3 298664.47]
 [0.0 0.0 1.0 78389.47 153773.43 299737.29]
 [0.0 1.0 0.0 73994.56 122782.75 303319.26]
 [0.0 1.0 0.0 67532.53 105751.03 304768.73]
 [0.0 0.0 1.0 77044.01 99281.34 140574.81]
 [1.0 0.0 0.0 64664.71 139553.16 137962.62]
 [0.0 1.0 0.0 75328.87 144135.98 134050.07]
 [0.0 0.0 1.0 72107.6 127864.55 353183.81]
 [0.0 1.0 0.0 66051.52 182645.56 118148.2]
 [0.0 0.0 1.0 65605.48 153032.06 107138.38]
 [0.0 1.0 0.0 61994.48 115641.28 91131.24]
 [0.0 0.0 1.0 61136.38 152701.92 88218.23]
 [1.0 0.0 0.0 63408.86 129219.61 46085.25]
 [0.0 1.0 0.0 55493.95 103057.49 214634.81]
 [1.0 0.0 0.0 46426.07 157693.92 210797.67]
 [0.0 0.0 1.0 46014.02 85047.44 205517.64]
 [0.0 1.0 0.0 28663.76 127056.21 201126.82]
 [1.0 0.0 0.0 44069.95 51283.14 197029.42]
 [0.0 0.0 1.0 20229.59 65947.93 185265.1]
 [1.0 0.0 0.0 38558.51 82982.09 174999.3]
 [1.0 0.0 0.0 28754.33 118546.05 172795.67]
 [0.0 1.0 0.0 27892.92 84710.77 164470.71]
 [1.0 0.0 0.0 23640.93 96189.63 148001.11]
 [0.0 0.0 1.0 15505.73 127382.3 35534.17]
 [1.0 0.0 0.0 22177.74 154806.14 28334.72]
 [0.0 0.0 1.0 1000.23 124153.04 1903.93]
 [0.0 1.0 0.0 1315.46 115816.21 297114.46]
 [1.0 0.0 0.0 0.0 135426.92 0.0]
 [0.0 0.0 1.0 542.05 51743.15 0.0]
 [1.0 0.0 0.0 0.0 116983.8 45173.06]]
[[103015.2  103282.38]
 [132582.28 144259.4 ]
 [132447.74 146121.95]
 [ 71976.1   77798.83]
 [178537.48 191050.39]
 [116161.24 105008.31]
 [ 67851.69  81229.06]
 [ 98791.73  97483.56]
 [113969.44 110352.25]
 [167921.07 166187.94]]
Predicted Profit	Actual Profit
----------------------------------
103,015.20		103,282.38
132,582.28		144,259.40
132,447.74		146,121.95
71,976.10		77,798.83
178,537.48		191,050.39
116,161.24		105,008.31
67,851.69		81,229.06
98,791.73		97,483.56
113,969.44		110,352.25
167,921.07		166,187.94

Simple linear regression

2025-04-18T09:21:00.000-07:00

Simple linear regression:-

There is only one continuous independent variable x and the assumed relation between the independent variable and the dependent variable y is

y = a + bx.

Simple linear regression is an approach for predicting a response using a single feature. It is assumed that the two variables are linearly related. Hence, we try to find a linear function that predicts the response value(y) as accurately as possible as a function of the feature or independent variable(x).

Let us start to experiment with Simple Linear Regression:

Download the dataset:

Code:

import numpy as np

import pandas as pd

import matplotlib.pyplot as plt

## load the dataset

dataset = pd.read_csv("C:\\Users\\SR Laptop\\Desktop\\Linear regression\\Salary_dataset.csv")

X =dataset.iloc[:, 0:1].values

y = dataset.iloc[:, -1].values

##Spliting the datasets into training and testing test

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test= train_test_split(X,y, test_size=0.7, random_state=0)

print("features YOE: ", X_train)

print(" ")

print("Label Salary: ", y_train)

print(" ")

print("features YOE: ", X_test)

print(" ")

print("Label Salary: ", y_test)

print(" ")

#train the simple linear regression model on the training sets

from sklearn.linear_model import LinearRegression

regressor= LinearRegression()

regressor.fit(X_train, y_train)

print("features YOE: ", X_train)

print(" ")

print("Label Salary: ", y_train)

print(" ")

print("features YOE: ", X_test)

print(" ")

print("Label Salary: ", y_test)

print(" ")

##predict the test sets results

y_pred = regressor.predict(X_test)

##visualizing the training set results

plt.scatter(X_train,y_train,color='r')

plt.plot(X_train, regressor.predict (X_train), color='b')

plt.title('salary vs experince(training set)')

plt.xlabel('years of exprence')

plt.ylabel('salary')

plt.show()

##Visualizing the test set Results

plt.scatter(X_test,y_test,color='r')

plt.plot(X_train, regressor.predict (X_train), color='b')

plt.title('salary vs experince(testing set)')

plt.xlabel('years of exprence')

plt.ylabel('salary')

plt.show()

##visulization with predicted values

plt.scatter(X_test,y_test,color='r')

plt.plot(X_test, y_pred, color='b')

plt.title('salary vs experince(predicted test values)')

plt.xlabel('years of exprence')

plt.ylabel('salary')

plt.show()

##make a single prediction

single_prediction= regressor.predict([[12]])

print(single_prediction)

##print the model parameters

coefficient= regressor.coef_

print("coefficient is:", coefficient)

intercept= regressor.intercept_

print("intercept is:", intercept)

##manual calculation of salary prediction extra part

manual_prediction= 2835.78327444 + 33603.2285041225*15

print("manual prediction by using coeficient and intercept: ", manual_prediction)

#auto prediction

auto_prediction= regressor.predict([[15]])

print("Auto regression is: ", auto_prediction)

Output:

features YOE: [[19]

[ 9]

[ 7]

[25]

[ 3]

[ 0]

[21]

[15]

[12]]

Label Salary: [ 93941. 57190. 54446. 105583. 43526. 39344. 98274. 67939. 56958.]

features YOE: [[ 2]

[28]

[13]

[10]

[26]

[24]

[27]

[11]

[17]

[22]

[ 5]

[16]

[ 8]

[14]

[23]

[20]

[ 1]

[29]

[ 6]

[ 4]

[18]]

Label Salary: [ 37732. 122392. 57082. 63219. 116970. 109432. 112636. 55795. 83089.

101303. 56643. 66030. 64446. 61112. 113813. 91739. 46206. 121873.

60151. 39892. 81364.]

features YOE: [[19]

[ 9]

[ 7]

[25]

[ 3]

[ 0]

[21]

[15]

[12]]

Label Salary: [ 93941. 57190. 54446. 105583. 43526. 39344. 98274. 67939. 56958.]

features YOE: [[ 2]

[28]

[13]

[10]

[26]

[24]

[27]

[11]

[17]

[22]

[ 5]

[16]

[ 8]

[14]

[23]

[20]

[ 1]

[29]

[ 6]

[ 4]

[18]]

Label Salary: [ 37732. 122392. 57082. 63219. 116970. 109432. 112636. 55795. 83089.

101303. 56643. 66030. 64446. 61112. 113813. 91739. 46206. 121873.

60151. 39892. 81364.]

[67632.62779741]

coefficient is: [2835.78327444]

intercept is: 33603.2285041225

manual prediction by using coeficient and intercept: 506884.21083627746

Auto regression is: [76139.97762073]

Clustering and Association

2025-04-17T09:29:00.000-07:00

Clustering

Clustering is the process of grouping data points into clusters based on their similarity. This technique is useful for identifying patterns and relationships in data without the need for labeled examples. Cluster analysis finds the commonalities between the data objects and categorizes them as per the presence and absence of those commonalities.

Here are some clustering algorithms:

· K-Means Clustering algorithm

· Mean-shift algorithm

· DBSCAN Algorithm

· Principal Component Analysis

· Independent Component Analysis

Association:

Association learning is a machine learning method for discovering interesting relations, called “association rules”, between variables in large databases using some measures of “interestingness”.

It determines the set of items that occurs together in the dataset. Association rule makes marketing strategy more effective.

Such as people who buy X item (suppose a bread) are also tend to purchase Y (Butter/Jam) item. A typical example of Association rule is Market Basket Analysis.

Example

Consider a supermarket chain. The management of the chain is interested in knowing whether there are any patterns in the purchases of products by customers like the following:

“If a customer buys onions and potatoes together, then he/she is likely to also buy hamburger.”

From the standpoint of customer behaviour, this defines an association between the set of products {onion, potato} and the set {burger}. This association is represented in the form of a rule as follows:

{ onion, potato} => burger

The measure of how likely a customer, who has bought onion and potato, to buy burger also is given by the conditional probability

P ((onion, potato}|(burger}).

If this conditional probability is 0.8, then the rule may be stated more precisely as follows: “80% of customers who buy onion and potato also buy burger.”

Algorithms

There are several algorithms for generating association rules. Some of the well-known algorithms are listed below:

a) Apriori algorithm

b) Eclat algorithm

c) FP-Growth Algorithm (FP stands for Frequency Pattern)

Classification and Regression

2025-04-17T04:24:00.000-07:00

Classification

In machine learning, classification is the problem of identifying to which of a set of categories a new observation belongs, on the basis of a training set of data containing observations (or instances) whose category membership is known.

Example

Consider the following data:

Score1	29	22	10	31	17	33	32	20
Score2	43	29	47	55	18	54	40	41
Result	Pass Fail Fail Pass Fail Pass Pass Pass

Table 1.1: Example data for a classification problem

Data in Table 1.1 is the training set of data. There are two attributes “Score1” and “Score2”. The class label is called “Result”. The class label has two possible values “Pass” and “Fail”. The data can be divided into two categories or classes: The set of data for which the class label is “Pass” and the set of data for which the class label is “Fail”.

Let us assume that we have no knowledge about the data other than what is given in the table. Now, the problem can be posed as follows:

If we have some new data, say “Score1 = 25” and “Score2 = 36”, what value should be assigned to “Result” corresponding to the new data; in other words, to which of the two categories or classes the new observation should be assigned? See Figure 1.1: for a graphical representation of the problem.

Figure 1.1: Graphical representation of data in Table 1.1. Solid dots represent data in “Pass” class and hollow dots data in “Fail” class. The class label of the square dot is to be determined.

To answer this question, using the given data alone we need to find the rule, or the formula, or the method that has been used in assigning the values to the class label “Result”. The problem of finding this rule or formula or the method is the classification problem. In general, even the general form of the rule or function or method will not be known. So several different rules, etc. may have to be tested to obtain the correct rule or function or method.

Examples

Consider the data given in Table 1.1 and the associated classification problem. We may consider the following rules for the classification of the new data:

IF Score1 + Score2 ≥ 60, THEN “Pass” ELSE “Fail”.

IF Score1 ≥ 20 AND Score2 ≥ 40 THEN “Pass” ELSE “Fail”.

Or, we may consider the following rules with unspecified values for M, m₁, m₂ and then by some method estimate their values.

IF Score1 + Score2 ≥ M, THEN “Pass” ELSE “Fail”.

IF Score1 ≥ m₁ AND Score2 ≥ m₂ THEN “Pass” ELSE “Fail”.

Algorithms

There are several machine learning algorithms for classification. The following are some of the well-known algorithms.

a) Logistic regression

b) Naive Bayes algorithm

c) k-NN algorithm

d) Decision tree algorithm

e) Support vector machine algorithm

f) Random forest algorithm

Remarks

• A classification problem requires that examples be classified into one of two or more classes.

• A classification can have real-valued or discrete input variables.

• A problem with two classes is often called a two-class or binary classification problem.

• A problem with more than two classes is often called a multi-class classification problem.

• A problem where an example is assigned multiple classes is called a multi-label

classification problem.

Regression

In machine learning, a regression problem is the problem of predicting the value of a numeric variable based on observed values of the variable. The value of the output variable may be a number, such as an integer or a floating point value. These are often quantities, such as amounts and sizes. The input variables may be discrete or real-valued.

Example

Consider the data on car prices given in Table 1.2.

Price	Age	Distance	Weight
(US$)	(years)	(KM)	(pounds)
13500	23	46986	1165
13750	23	72937	1165
13950	24	41711	1165
14950	26	48000	1165
13750	30	38500	1170
12950	32	61000	1170
16900	27	94612	1245
18600	30	75889	1245
21500	27	19700	1185
12950	23	71138	1105

Table 1.2: Prices of used cars: example data for regression

Suppose we are required to estimate the price of a car aged 25 years with distance 53240 KM and weight 1200 pounds. This is an example of a regression problem because we have to predict the value of the numeric variable “Price”.

General approach

Let x denote the set of input variables and y the output variable. In machine learning, the general approach to regression is to assume a model, that is, some mathematical relation between x and y, involving some parameters say, θ, in the following form:

y = f (x, θ)

The function y = f (x, θ) is called the regression function. The machine learning algorithm optimizes the parameters in the set θ such that the approximation error is minimized; that is, the estimates of the values of the dependent variable y are as close as possible to the correct values given in the training set.

Example

For example, if the input variables are “Age”, “Distance” and “Weight” and the output variable is “Price”, the model may be

y = f (x, θ)

Price = a₀ + a₁ × (Age) + a₂ × (Distance) + a₃ × (Weight)

Where x = (Age, Distance, Weight) denotes the set of input variables and θ = (a₀, a₁, a₂, a₃) denotes the set of parameters of the model.

regression Algorithms

• Simple linear regression

• Multivariate linear regression

• Polynomial regression

• Logistic regression

• Decision tree

• Random forest

Different regression models:

There are various types of regression techniques available to make predictions. These techniques mostly differ in three aspects, namely, the number and type of independent variables, the type of dependent variables and the shape of regression line. Some of these are listed below.

• Simple linear regression: There is only one continuous independent variable x and the assumed relation between the independent variable and the dependent variable y is

y = a + bx.

• Multivariate linear regression: There are more than one independent variable, say x₁, . . . , x_n, and the assumed relation between the independent variables and the dependent variable is

y = a₀ + a₁x₁ + ⋯ + a_nx_n.

• Polynomial regression: There is only one continuous independent variable x and the assumed model is

y = a₀ + a₁x + ⋯ + a_nxⁿ.

• Logistic regression: The dependent variable is binary, that is, a variable which takes only the values 0 and 1. The assumed model involves certain probability distributions.

Naive Bayes classifier

2025-04-17T02:24:00.000-07:00

Naive Bayes is a statistical classification technique based on Bayes Theorem. It is one of the simplest supervised learning algorithms. Naive Bayes classifier is the fast, accurate and reliable algorithm. Naive Bayes classifiers have high accuracy and speed on large datasets.

Naive Bayes classifier assumes that the effect of a particular feature in a class is independent of other features. For example, a loan applicant is desirable or not depending on his/her income, previous loan and transaction history, age, and location. Even if these features are interdependent, these features are still considered independently. This assumption simplifies computation, and that's why it is considered as naive. This assumption is called class conditional independence.

· P(h): the probability of hypothesis h being true (regardless of the data). This is known as the prior probability of h.

· P(D): the probability of the data (regardless of the hypothesis). This is known as the prior probability.

· P(h|D): the probability of hypothesis h given the data D. This is known as posterior probability.

· P(D|h): the probability of data d given that the hypothesis h was true. This is known as posterior probability.

Let’s understand the concept of the Naive Bayes Theorem through an example. We are taking a dataset of employees in a company, our aim is to create a model to find whether a person is going to the office by driving or walking using salary and age of the person.

In the above, we can see 30 data points in which red points belong to those who are walking and green belongs to those who are driving. Now let’s add a new data point into it. Our aim is to find the category that the new point belongs to.

Note that we are taken age on the X-axis and Salary on the Y-axis. We are using the Naive Bayes algorithm to find the category of the new data point. For this, we have to find the posterior probability of walking and driving for this data point. After comparing, the point belongs to the category having a higher probability.

The posterior probability of walking for the new data point is :

also for the driving is :

Naive Bayes algorithm

Step 1: We have to find all the probabilities required for the Bayes theorem for the calculation of posterior probability

P(Walks) is simply the probability of those who walk among all

In order to find the marginal likelihood, P(X), we have to consider a circle around the new data point of any radii including some red and green points.

P(X|Walks) can be found by :

Now we can find the posterior probability using the Bayes theorem,

Step 2: Similarly we can find the posterior probability of Driving, and it is 0.25

Step 3: Compare both posterior probabilities. When comparing the posterior probability, we can find that P(walks|X) has greater values and the new point belongs to the walking category.

Types of Machine Learning

2025-04-15T01:47:00.000-07:00

In general, machine learning algorithms can be classified into three types.

(1) Supervised learning

(2) Unsupervised learning

(3) Reinforcement learning

Additionally, there is a more specific category called semi-supervised learning, which

combines elements of both supervised and unsupervised learning.

Figure 1: Categories of machine learning

1. Supervised Learning:-

Supervised learning is the machine learning task of learning a function that maps an input to an output based on example input-output pairs.

In supervised learning, each example in the training set is a pair consisting of an input object (typically a vector) and an output value. A supervised learning algorithm analyzes the training data and produces a function, which can be used for mapping new examples. In the optimal case, the function will correctly determine the class labels for unseen instances. Both classification and regression problems are supervised learning problems.

A wide range of supervised learning algorithms are available, each with its strengths and weak- nesses. There is no single learning algorithm that works best on all supervised learning problem.

Figure 1.4: Supervised learning

There are two main categories of supervised learning that are:

· Classification

· Regression

Supervised learning is the subcategory of machine learning that focuses on learning a classification (Figure 2), or regression model (Figure 3), that is, learning from labeled training data (i.e., inputs that also contain the desired outputs or targets; basically, “examples” of what we want to predict).

Figure 2: Illustration of a binary classification problem (plus and minus signs denote class labels) and two feature variables, (x1 and x2). (Source: Raschka & Mirjalili: Python Machine Learning, 3rd Ed.).

Figure 3: Illustration of a linear regression model with one feature variable (x1) and the target variable y. The dashed-line indicates the functional form of the linear regression model. (Source: Raschka & Mirjalili: Python Machine Learning, 3rd Ed.).

Example

Consider the following data regarding patients entering a clinic. The data consists of the gender and age of the patients and each patient is labeled as “healthy” or “sick”.

gender	age	label
M	48	sick
M	67	sick
F	53	healthy
M	49	healthy
F	34	sick
M	21	healthy

Based on this data, when a new patient enters the clinic, how can one predict whether he/she is healthy or sick?

Advantages of Supervised Machine Learning:-

· Supervised Learning models can have high accuracy as they are trained on labelled data.

· The process of decision-making in supervised learning models is often interpretable.

· It can often be used in pre-trained models which saves time and resources when developing new models from scratch.

Disadvantages of Supervised Machine Learning:-

· It has limitations in knowing patterns and may struggle with unseen or unexpected patterns that are not present in the training data.

· It can be time-consuming and costly as it relies on labeled data only.

· It may lead to poor generalizations based on new data.

Applications of Supervised Learning:-

Supervised learning is used in a wide variety of applications, including:

· Image classification: Identify objects, faces, and other features in images.

· Natural language processing: Extract information from text, such as sentiment, entities, and relationships.

· Speech recognition: Convert spoken language into text.

· Recommendation systems: Make personalized recommendations to users.

· Predictive analytics: Predict outcomes, such as sales, customer churn, and stock prices.

· Medical diagnosis: Detect diseases and other medical conditions.

· Fraud detection: Identify fraudulent transactions.

· Autonomous vehicles: Recognize and respond to objects in the environment.

· Email spam detection: Classify emails as spam or not spam.

· Quality control in manufacturing: Inspect products for defects.

· Credit scoring: Assess the risk of a borrower defaulting on a loan.

· Gaming: Recognize characters, analyze player behavior, and create NPCs.

· Customer support: Automate customer support tasks.

· Weather forecasting: Make predictions for temperature, precipitation, and other meteorological parameters.

· Sports analytics: Analyze player performance, make game predictions, and optimize strategies.

2. Unsupervised learning:-

Unsupervised learning is a type of machine learning algorithm used to draw inferences from datasets consisting of input data without labeled responses.

In unsupervised learning algorithms, a classification or categorization is not included in the observations. There are no output values and so there is no estimation of functions. Since the examples given to the learner are unlabeled, the accuracy of the structure that is output by the algorithm cannot be evaluated.

The most common unsupervised learning method is cluster analysis, which is used for ex- ploratory data analysis to find hidden patterns or grouping in data.

Example

Consider the following data regarding patients entering a clinic. The data consists of the gender and age of the patients.

gender	age
M	48
M	67
F	53
M	49
F	34
M	21

Based on this data, can we infer anything regarding the patients entering the clinic?

There are two main categories of unsupervised learning that are:

· Clustering

· Association

In contrast to supervised learning, unsupervised learning is a branch of machine learning that is concerned with unlabeled data. Common tasks in unsupervised learning are clustering analysis (assigning group memberships; Figure 4) and dimensionality reduction (compressing data onto a lower-dimensional subspace or manifold).

Figure 4: Illustration of clustering, where the dashed lines indicate potential group membership assignments of unlabeled data points. (Source: Raschka & Mirjalili: Python Machine Learning, 3rd Ed.).

Advantages of Unsupervised Machine Learning:-

· It helps to discover hidden patterns and various relationships between the data.

· Used for tasks such as customer segmentation, anomaly detection, and data exploration.

· It does not require labeled data and reduces the effort of data labeling.

Disadvantages of Unsupervised Machine Learning:-

· Without using labels, it may be difficult to predict the quality of the model’s output.

· Cluster Interpretability may not be clear and may not have meaningful interpretations.

· It has techniques such as autoencoders and dimensionality reduction that can be used to extract meaningful features from raw data.

Applications of Unsupervised Learning:-

Here are some common applications of unsupervised learning:

Clustering: Group similar data points into clusters.
Anomaly detection: Identify outliers or anomalies in data.
Dimensionality reduction: Reduce the dimensionality of data while preserving its essential information.
Recommendation systems: Suggest products, movies, or content to users based on their historical behavior or preferences.
Topic modeling: Discover latent topics within a collection of documents.
Density estimation: Estimate the probability density function of data.Image and video compression: Reduce the amount of storage required for multimedia content.
Data preprocessing: Help with data preprocessing tasks such as data cleaning, imputation of missing values, and data scaling.
Market basket analysis: Discover associations between products.
Genomic data analysis: Identify patterns or group genes with similar expression profiles.
Image segmentation: Segment images into meaningful regions.
Community detection in social networks: Identify communities or groups of individuals with similar interests or connections.
Customer behavior analysis: Uncover patterns and insights for better marketing and product recommendations.
Content recommendation: Classify and tag content to make it easier to recommend similar items to users.
Exploratory data analysis (EDA): Explore data and gain insights before defining specific tasks.

3. Reinforcement learning:-

Reinforcement is the process of learning from rewards while performing a series of actions. In reinforcement learning, we do not tell the learner or agent (for example, a robot), which action to take but merely assign a reward to each action and/or the overall outcome. Instead of having “correct/false” labels for each step, the learner must discover or learn a behavior that maximizes the reward for a series of actions. In that sense, it is not a supervised setting. RL is somewhat related to unsupervised learning; however, reinforcement learning really is its own category of machine learning. Reinforcement learning will not be covered further in this class. However, for those who are interested, Dr. Mirjalili and I wrote an introduction to reinforcement learning for the 3rd edition of “Python Machine Learning.”

Typical applications of reinforcement learning involve playing games (chess, Go, Atari video games) and some form of robots, e.g., drones, warehouse robots, and more recently self- driving cars.

Example:

Consider teaching a dog a new trick: we cannot tell it what to do, but we can reward/punish it if it does the right/wrong thing. It has to find out what it did that made it get the reward/punishment. We can use a similar method to train computers to do many tasks, such as playing backgammon or chess, scheduling jobs, and controlling robot limbs. Reinforcement learning is different from supervised learning. Supervised learning is learning from examples provided by a knowledgeable expert.

Figure 5: Illustration of reinforcement learning (Source: Raschka & Mirjalili: Python Machine Learning, 3rd Ed.).

Advantages of Reinforcement Machine Learning

It has autonomous decision-making that is well-suited for tasks and that can learn to make a sequence of decisions, like robotics and game-playing.

This technique is preferred to achieve long-term results that are very difficult to achieve.

It is used to solve a complex problems that cannot be solved by conventional techniques.

Disadvantages of Reinforcement Machine Learning

Training Reinforcement Learning agents can be computationally expensive and time-consuming.

Reinforcement learning is not preferable to solving simple problems.

It needs a lot of data and a lot of computation, which makes it impractical and costly.

Applications of Reinforcement Machine Learning

Here are some applications of reinforcement learning:

Game Playing: RL can teach agents to play games, even complex ones.

Robotics: RL can teach robots to perform tasks autonomously.

Autonomous Vehicles: RL can help self-driving cars navigate and make decisions.

Recommendation Systems: RL can enhance recommendation algorithms by learning user preferences.

Healthcare: RL can be used to optimize treatment plans and drug discovery.

Natural Language Processing (NLP): RL can be used in dialogue systems and chatbots.

Finance and Trading: RL can be used for algorithmic trading.

Supply Chain and Inventory Management: RL can be used to optimize supply chain operations.

Energy Management: RL can be used to optimize energy consumption.

Game AI: RL can be used to create more intelligent and adaptive NPCs in video games.

Adaptive Personal Assistants: RL can be used to improve personal assistants.

Virtual Reality (VR) and Augmented Reality (AR): RL can be used to create immersive and interactive experiences.

Industrial Control: RL can be used to optimize industrial processes.

Education: RL can be used to create adaptive learning systems.

Agriculture: RL can be used to optimize agricultural operations.

4. Semi-supervised learning:-

Loosely speaking, semi-supervised learning can be described as a mix between supervised and unsupervised learning. In semi-supervised learning tasks, some training examples contain outputs, but some do not. We then use the labeled training subset to label the unlabeled portion of the training set, which we then also utilize for model training.

Machine Learning Model

2025-04-15T00:49:00.000-07:00

Machine Learning key terms that you must know!

Features:

• Features are the fields used as input.

• A feature is one column of the data in your input set.

• For instance, if you're trying to predict the type of pet someone will choose, your input features might include age, home region, family income, etc.

• Feature means property of your training data.

• A feature is the input you have fed to the model or system.

• The value of x variable in simple linear regression are the features.

Label:

• The output you get from your model after training is called a label.

• A label is the thing we're predicting.

• For example the value of y variable in simple linear regression model is the label.

• Suppose you give your model data like a person’s age, height, hair length and then your model predicts whether the person is male or female. Then male or female is called the label.

Machine Learning Model:

• A model is the relationship between features and the label.

• An ML model is a mathematical model that generates predictions by finding patterns in your data.

• ML Models generate predictions using the patterns extracted from the input data.

• A model represents what was learned by a machine learning algorithm.

• The model is the “thing” that is saved after running a machine learning algorithm on training data and represents the rules, numbers, and any other algorithm-specific data structures required to make predictions.

1. Data collection:

Data collection is the process of gathering and measuring information from countless different sources. This is a critical first step that involves gathering data from various sources such as databases, files, and external repositories. Before starting the data collection process, it’s important to articulate the problem you want to solve with an ML model.

2. Data Preparation:

Data preparation/pre-processing techniques generally refer to the addition, deletion, or transformation of training set data. Since the collected data may be in an undesired format, unorganized, or extremely large, further steps are needed to enhance its quality. The three common steps for preprocessing data are

· formatting,

· cleaning,

· and sampling.

Data preparation (also referred to as “data preprocessing”) is the process of transforming raw data so that data scientists and analysts can run it through machine learning algorithms to uncover insights or make predictions.

3. Choose a ML model:

For different purpose, different ML models are available. So it depends on the need that which ML model must be selected. The choice of ML model to be selected depends on many factors like the problem statement and the kind of output you want, type and size of the data, the available computational time, number of features, and observations in the data, etc.

4. Train the model:

The process of training an ML model involves providing an ML algorithm (that is, the learning algorithm) with training data to learn from. Let's say that you want to train an ML model to predict if an email is spam or not spam. You would provide ML model with training data that contains emails for which you know the target (that is, a label that tells whether an email is spam or not spam). Then the model should be trained by using this data, resulting in a model that attempts to predict whether new email will be spam or not spam.

5. Evaluate the model:

Model evaluation is a method of assessing the correctness of models on test data. The test data consists of data points that have not been seen by the model before. There are two methods of evaluating models in data science, Hold-Out and Cross-Validation. To avoid overfitting, both methods use a test set (not seen by the model) to evaluate model performance.

6. Parameter Tuning:

Each model has its own sets of parameters that need to be tuned to get optimal output. For every model, our goal is to minimize the error or say to have predictions as close as possible to actual values. This is one of the cores or say the major objective of hyperparameter tuning. There are following three approaches to Hyperparameter tuning:

•Manual Search

•Random Search

•Grid Search

7. Make predictions:

“Prediction” refers to the output of an algorithm after it has been trained on a historical dataset. Machine learning has two main goals:

· prediction and inference.

After you have a model, you can use that model to generate predictions which means to give your model the inputs it has never seen before and obtain the answer the model has predicted. In addition to making predictions on new data, you can use machine-learning models to better understand the relationships between the input features and the output target which is known as inference.

Understanding data

2025-04-15T00:31:00.000-07:00

Understanding data

Since an important component of the machine learning process is data storage, we briefly consider in this section the different types and forms of data that are encountered in the machine learning process.

1. Unit of observation

By a unit of observation we mean the smallest entity with measured properties of interest for a study.

Examples

• A person, an object or a thing

• A time point

• A geographic region

• A measurement

Sometimes, units of observation are combined to form units such as person-years.

2. Examples and features

Datasets that store the units of observation and their properties can be imagined as collections of data consisting of the following:

· Examples

An “example” is an instance of the unit of observation for which properties have been recorded.

An “example” is also referred to as an “instance”, or “case” or “record.” (It may be noted that

the word “example” has been used here in a technical sense.)

· Features

A “feature” is a recorded property or a characteristic of examples. It is also referred to as

“attribute”, or “variable” or “feature.”

Examples for “examples” and “features”

1. Cancer detection

Consider the problem of developing an algorithm for detecting cancer. In this study we note

the following.

(a) The units of observation are the patients.

(b) The examples are members of a sample of cancer patients.

• gender

• age

• blood pressure

• the findings of the pathology report after a biopsy

2. Pet selection

Suppose we want to predict the type of pet a person will choose.

(a) The units are the persons.

(b) The examples are members of a sample of persons who own pets.

pets.

Figure 1: Example for “examples” and “features” collected in a matrix format (data relates to automobiles and their features)

3. Spam e-mail

Let it be required to build a learning algorithm to identify spam e-mail.

(a) The unit of observation could be an e-mail message.

(b) The examples would be specific messages.

Examples and features are generally collected in a “matrix format”. Fig. 1: shows such a dataset.

1. Different forms of data

1. Numeric data

If a feature represents a characteristic measured in numbers, it is called a numeric feature.

2. Categorical or nominal

A categorical feature is an attribute that can take on one of a limited, and usually fixed, number of possible values on the basis of some qualitative property. A categorical feature is also called a nominal feature.

3. Ordinal data

This denotes a nominal variable with categories falling in an ordered list. Examples include clothing sizes such as small, medium, and large, or a measurement of customer satisfaction on a scale from “not at all happy” to “very happy.”

Examples

In the data given in Fig.1, the features “year”, “price” and “mileage” are numeric and the features “model”, “color” and “transmission” are categorical.

Learning

2025-04-15T00:24:00.000-07:00

Definition of learning:

Definition:

A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks T, as measured by P, improves with experience E.

Examples:

i) Handwriting recognition learning problem

• Task T: Recognising and classifying handwritten words within images

• Performance P: Percent of words correctly classified

• Training experience E: A dataset of handwritten words with given classifications

ii) A robot driving learning problem

• Task T: Driving on highways using vision sensors

• Performance measure P: Average distance traveled before an error

• training experience: A sequence of images and steering commands recorded while observing a human driver

iii) A chess learning problem

• Task T: Playing chess

• Performance measure P: Percent of games won against opponents

• Training experience E: Playing practice games against itself

Definition

A computer program which learns from experience is called a machine learning program or simply a learning program. Such a program is sometimes also referred to as a learner.

Components of Learning

Basic components of learning process

The learning process, whether by a human or a machine, can be divided into four components, namely, data storage, abstraction, generalization and evaluation.

Figure 1.1 illustrates the various components and the steps involved in the learning process.

1. Data storage

Facilities for storing and retrieving huge amounts of data are an important component of the learning process. Humans and computers alike utilize data storage as a foundation for advanced reasoning.

• In a human being, the data is stored in the brain and data is retrieved using electrochemical signals.

• Computers use hard disk drives, flash memory, random access memory and similar devices to store data and use cables and other technology to retrieve data.

2. Abstraction

The second component of the learning process is known as abstraction.

Abstraction is the process of extracting knowledge about stored data. This involves creating general concepts about the data as a whole. The creation of knowledge involves application of known models and creation of new models.

The process of fitting a model to a dataset is known as training. When the model has been trained, the data is transformed into an abstract form that summarizes the original information.

3. Generalization

The third component of the learning process is known as generalisation.

The term generalization describes the process of turning the knowledge about stored data into a form that can be utilized for future action. These actions are to be carried out on tasks that are similar, but not identical, to those what have been seen before. In generalization, the goal is to discover those properties of the data that will be most relevant to future tasks.

4. Evaluation

Evaluation is the last component of the learning process.

It is the process of giving feedback to the user to measure the utility of the learned knowledge. This feedback is then utilised to effect improvements in the whole learning process

Introduction to machine Learning

2025-04-14T23:36:00.000-07:00

What Is Machine Learning?

Machine learning is programming computers to optimize a performance criterion using example data or past experience. We have a model defined up to some parameters, and learning is the execution of a computer program to optimize the parameters of the model using the training data or past experience. The model may be predictive to make predictions in the future, or descriptive to gain knowledge from data, or both.

Arthur Samuel, an early American leader in the field of computer gaming and artificial intelligence, coined the term “Machine Learning” in 1959 while at IBM. He defined machine learning as “the field of study that gives computers the ability to learn without being explicitly programmed.” However, there is no universally accepted definition for machine learning. Different authors define the term differently.

Applications of machine learning

Application of machine learning methods to large databases is called data mining. In data mining, a large volume of data is processed to construct a simple model with valuable use, for example, having high predictive accuracy.

The following is a list of some of the typical applications of machine learning.

In retail business, machine learning is used to study consumer behaviour.
In finance, banks analyze their past data to build models to use in credit applications, fraud detection, and the stock market.
In manufacturing, learning models are used for optimization, control, and troubleshooting.
In medicine, learning programs are used for medical diagnosis.
In telecommunications, call patterns are analyzed for network optimization and maximizing the quality of service.
In science, large amounts of data in physics, astronomy, and biology can only be analyzed fast enough by computers. The World Wide Web is huge; it is constantly growing and searching for relevant information cannot be done manually.
In artificial intelligence, it is used to teach a system to learn and adapt to changes so that the system designer need not foresee and provide solutions for all possible situations.
It is used to find solutions to many problems in vision, speech recognition, and robotics.
Machine learning methods are applied in the design of computer-controlled vehicles to steer correctly when driving on a variety of roads.
Machine learning methods have been used to develop programmes for playing games such as chess, backgammon and Go.

Machine Learning

2025-04-14T23:24:00.000-07:00

This is an introductory Machine Learning course for beginners.

Introduction to Machine Learning
Learning
Understanding data
Machine Learning Model
Types of Machine Learning
Classification and Regression
Clustering and Association
Simple linear regression
Multiple Linear Regression
Naive Bayes classifier
Decision Tree

EXPLORING COMPUTER AND THEIR USES

2025-03-18T21:25:00.000-07:00

OVERVIEW:

“Computers are everywhere”. In the past two decades, computers have reshaped our lives at home, work and school. As your knowledge of computer grows, you will understand that all computers regardless of their sizes and purpose are similar.

THE COMPUTER DEFINED:

In basic terms, a computer is an electronic device that process data, converting it into information that is useful to people. Any computer regardless of its types is controlled by programming instructions, which gives the machine instructions and tell it what to do. A less known type of computer is ANALOG COMPUTER, which works in very different way as compared to digital computer. Analog and digital computer are different in many respects, but the most important thing they both represent is DATA. DIGITAL computer represent data as values with no other possibilities but the ANALOG computer represents data as variable and continuous values with other possibilities also. This makes analog computers more reliable and easy then digital computers. The common type of analog computer is old fashioned slide rule.

COMPUTER FOR INDIVIDUAL USERS:

Most computers are used by only person at a time. The six primary types of computers in this category are:

Desktop computer
Workstations
Tablet computers
Notebook computers
Handheld computers
Smartphones

These systems are all examples of personal computers(PCs). Personal computers are also called “Microcomputers”. The term personal computer PC is also used to describe desktop computers.

DESKTOP COMPUTERS:

The most common type of personal computers are “desktop computers”, that are designed to sit on a desk or a table. These are the systems that you see all around in your schools, offices, universities, working labs, homes and they are the main topic of this book. They can be used to communicate, produce music, edit photographs and videos, play games and much more. The main computer of the desktop PC is the system unit. There are two common designs for the desktop computers i.e, a horizontally oriented systems and a vertically oriented tower model.

WORKSTATIONS:

A Workstation is a specialized, single user computer that typically has more power and features then a standard desktop PC. Workstations have large, high resolutionized monitors, accelerated graphics and handling capabilities.

NOTEBOOK COMPUTERS:

Notebook computers having shape of an 8.5 by 11 inch notebook and easily put inside a briefcase. They are also called LAPTOP computers. Notebook computers are fully functional microcomputers. Because of their portability, notebook computer are fall into a device called MOBILE computers systems small enough to carried by their users.

TABLET PCs:

The tablet pcs are the newest development, full-featured computers. The tablet pcs are same as notebook pcs, but they are lighter and accept inputs from a pen called a STYLUS or a digital pen that is used to tap and write directly on the screen.

HANDHELD COMPUTERS:

Handheld computers are small enough to be fit in your hands. A popular type of Handheld computers is PERSONAL DIGITAL ASSISTANT PDA. A PDA can be connected to large computers for exchanging data.

SMARTPHONES:

A smartphone is a phone that performs many functions of the computers. They have functions include web and email access, special software such as personal organizers or special hardware such as digital cameras or music players.

COMPUTERS FOR ORGANIZATION:

Some computers handle the need of many users at a time, each user interact with the computer through his or her own device.

The largest organizational computers support thousands of the individual users at the same time. Powerful systems are often used by organizations.

Network servers
Mainframe computers
Minicomputers
Supercomputers

NETWORK SERVERS:

A network server is usually a powerful personal computer with special software and functions. PC based networks and servers offer companies a great deal of flexibility. Different servers have different purposes. A PC based server give user flexibility to do different kind of tasks. The servers can support many processes and many other while storing information and programs for many peoples to use.

MAINFRAME COMPUTERS:

Mainframe computers are use in large organizations such as banks and insurance companies where many people need to use the same data. Each user accesses the mainframe resources through a device called a TERMINAL. A dumb terminal (not storing or processing data) and an intelligent terminal (performing operations but not having storage usually). Mainframes are large powerful systems. Website to reserve a seat on a flight, you probably transaction with a mainframe computer.

MINICOMPUTERS:

Minicomputers got their names because of their small sized compared to the other computers of the day. The capabilities of a minicomputer are those between personal computers and mainframes. They are also called MIDRANGED computers.

SUPERCOMPUTERS:

Supercomputers are the most powerful computers and physically they are some of the largest. Supercomputers are ideal for handling large and highly complex problems that require extreme calculating power.

COMPUTERS IN SOCIETY:

Why computers are important in our society? People often talked about computer terms and their Impact on our lives. Computers are important in our society because information is so essential in our lives. All the information is stored and processed by computers. If you work in a place and need to perform various tasks, a desktop computer is the best choice. Choose a desktop computer if you want to work with graphics, designs or use multimedia products, setup complex hardware.

HOME:

People cannot imagine lives without computers. In fact, a growing families have multiple pcs in their homes. They use computers at home for communications, business work done, schoolwork, entertainment and finances.

EDUCATION:

More and more schools are adding computers technology in educational circle. Educators see computer technology as an essential learning requirement for all students. SMALL BUSSINESS: Many of todays successful companies are nothing without computers. Hundred of thousand of individuals launch business based from their home or in small office locations. Small business become more self-sufficient and reduce their operating expenses.

INDUSTRY:

Enterprises use different kinds of computers in many combinations. Computers are applied in industry for design, shipping and process control.

GOVERNMENT:

Computers plays important role in nearly every government agency. Such as in taxes, population, military, and police.

Application of Information & Communication Technology

2025-03-18T21:19:00.000-07:00

This is an introductory course in Computer Science designed for beginners. Apart from leading the participants through a whirlwind history of computing, the course also develops a feel for web programming through a series of lectures that help the students develop their own web page. Main objective of the course is to build an appreciation for the fundamental concepts in computing and to become familiar with popular PC productivity software.

This course, using both lecture and laboratory practice, introduces students to basic computer concepts in hardware, software, networking, computer security, database, e-commerce and other emerging technologies such as wiki and Google applications. Students learn techniques to search information found online. Widely used applications including word processing, spreadsheets, databases, presentation, and web development software are studied.

Course Objective:

After successful completion of the course, the students will be able to:-

Understand basic functions of computer hardware and software components including operating system functions

Identify various networks (LAN, WAN, intranet), topologies (ring, bus, star), protocols (TCP/IP, HTTP &DNS), media types and network hardware.

The importance of this course should motivate you all to take keen interest in the class proceedings and try to work hands on.

Design and create web pages using HTML

Understand the fundamentals of system analysis, life cycle of a program development.

Text books:

1. Introduction to computers, Peter Norton, 7^thEdition
2. Computer Science An Overview, J. Glenn Brookshear 13^th Edition.

EXPLORING COMPUTER AND THEIR USES

Encryption Algorithms

2023-11-15T13:10:00.000-08:00

Encryption Algorithms:-

Their are Symmetric Block Cipher Algorithms

DES (Data Encryption Standard)
3DES (Triple DES)
AES (Advanced Encryption Standard)

Data Encryption Standard (DES):-

The main standard for encrypting data was a symmetric algorithm known as the Data Encryption Standard (DES). However, this has now been replaced by a new standard known as the Advanced Encryption Standard (AES) which we will look at later.

DES is a 64 bit block cipher which means that it encrypts data 64 bits at a time. This is contrasted to a stream cipher in which only one bit at a time (or sometimes small groups of bits such as a byte) is encrypted.

History:-

DES was the result of a research project set up by International Business Machines (IBM) Corporation in the late 1960’s which resulted in a cipher known as LUCIFER. In the early 1970’s it was decided to commercialize LUCIFER and a number of significant changes were introduced. IBM was not the only one involved in these changes as they sought technical advice from the National Security Agency (NSA) (other outside consultants were involved but it is likely that the NSA were the major contributors from a technical point of view). The altered version of LUCIFER was put forward as a proposal for the new national encryption standard requested by the National Bureau of Standards (NBS)3 . It was finally adopted in 1977 as the Data Encryption Standard- DES . Some of the changes made to LUCIFER have been the subject of much controversy even to the present day. The most notable of these was the key size.

LUCIFER used a key size of 128 bits however this was reduced to 56 bits for DES. Even though DES actually accepts a 64 bit key as input, the remaining eight bits are used for parity checking and have no effect on DES’s security. Outsiders were convinced that the 56 bit key was an easy target for a brute force attack4 due to its extremely small size. The need for the parity checking scheme was also questioned without satisfying answers.

working:-

DES (and most of the other major symmetric ciphers) is based on a cipher known as the

Feistel block cipher. It consists of a number of rounds where each round contains bit-shuffling, non- linear substitutions (S-boxes) and exclusive OR operations.

As with most encryption schemes, DES expects two inputs - the plaintext to be encrypted and the secret key. The manner in which the plaintext is accepted, and the key arrangement used for encryption and decryption, both determine the type of cipher it is.

DES is therefore a symmetric, 64 bit block cipher as it uses the same key for both encryption and decryption and only operates on 64 bit blocks of data at a time5 (be they plaintext or ciphertext).

The key size used is 56 bits, however a 64 bit (or eight-byte) key is actually input. The least significant bit of each byte is either used for parity (odd for DES) or set arbitrarily and does not increase the security in any way. All blocks are numbered from left to right which makes the eight bit of each byte the parity bit.

Once a plain-text message is received to be encrypted, it is arranged into 64 bit blocks

required for input. If the number of bits in the message is not evenly divisible by 64, then the last block will be padded. Multiple permutations and substitutions are incorporated

throughout in order to increase the difficulty of performing a cryptanalysis on the cipher.

AES (ADVANCED ENCRYPTION ALGORITHM):-

AES is a block cipher with a block length of 128 bits.
AES allows for three different key lengths: 128, 192, or 256 bits.
Most of our discussion will assume that the key length is 128 bits.
Encryption consists of

10 rounds of processing for 128-bit keys,

12 rounds for 192-bit keys,

14 rounds for 256-bit keys.

Except for the last round in each case, all other rounds are identical. Each round of processing includes one single-byte base substitution step, a row- wise permutation step, a column-wise mixing step, and the addition of the round key. The order in which these four steps are executed is different for encryption and decryption.
• To appreciate the processing steps used in a single round, it is best to think of a 128-bit block as consisting of a 4 × 4 matrix of bytes, arranged as follows:

Therefore, the first four bytes of a 128-bit input block occupy the first column in the 4 × 4 matrix of bytes. The next four bytes occupy the second column, and so on.

The 4×4 matrix of bytes shown above is referred to as the state array in AES.

The algorithm begins with an Add round key stage followed by 9 rounds of four stages and a tenth round of three stages.
This applies for both encryption and decryption with the exception that each stage of a round the decryption algorithm is the inverse of its counterpart in the encryption algorithm.
The four stages are as follows: 1. Substitute bytes 2. Shift rows 3. Mix Columns 4. Add
Round Key.

Substitute Bytes:-

•This stage (known as Sub Bytes) is simply a table lookup using a 16 × 16 matrix of byte values called an s-box.
•This matrix consists of all the possible combinations of an 8 bit sequence (28 = 16 × 16 = 256).
•However, the s-box is not just a random permutation of these values and there is a well defined method for creating the s-box tables.
•The designers of Rijndael showed how this was done unlike the s-boxes in DES for which no rationale was given. Our concern will be how state is effected in each round.
•For this particular round each byte is mapped into a new byte in the following way: the leftmost nibble of the byte is used to specify a particular row of the s-box and the rightmost nibble specifies a column.
•For example, the byte {95} (curly brackets represent hex values in FIPS PUB 197) selects row 9 column 5 which turns out to contain the value {2A}.
• This is then used to update the state matrix.

Shift Row Transformation:-

• This stage (known as Shift Rows) is shown in figure below.
• Simple permutation an nothing more.
• It works as follow: – The first row of state is not altered. – The second row is shifted 1 bytes to the left in a circular manner. – The third row is shifted 2 bytes to the left in a circular manner. – The fourth row is shifted 3 bytes to the left in a circular manner.

MIX COLUMN TRANSFORMATION:-

• This stage (known as Mix Column) is basically a substitution

• Each column is operated on individually. Each byte of a column is mapped into a new value that is a function of all four bytes in the column.

• The transformation can be determined by the following matrix multiplication on state

• Each element of the product matrix is the sum of products of elements of one row and one column.

ADD ROUND KEY TRANSFORMATION:-

• In this stage (known as Add Round Key) the 128 bits of state are bitwise XORed with the 128 bits of the round key.
• The operation is viewed as a column wise operation between the 4 bytes of a state column and one word of the round key.
• This transformation is as simple as possible which helps in efficiency but it also effects every bit of state.
• The AES key expansion algorithm takes as input a 4-word key and produces a linear array of 44 words. Each round uses 4 of these words as shown in figure.
• Each word contains 32 bytes which means each subkey is 128 bits long. Figure 7 show pseudocode for generating the expanded key from the actual key.

Digital Signature

2023-11-11T05:48:00.001-08:00

Digital signature:-

The most important development from the work on public-key cryptography is the digital signature. Message authentication protects two parties who exchange messages from any third party. However, it does not protect the two parties against each other. A digital signature is analogous to the handwritten signature, and provides a set of security capabilities that would be difficult to implement in any other way. It must have the following properties:

It must verify the author and the date and time of the signature
It must to authenticate the contents at the time of the signature
It must be verifiable by third parties, to resolve disputes

Thus, the digital signature function includes the authentication function. A variety of approaches has been proposed for the digital signature function.

These approaches fall into two categories: direct and arbitrated.

Operation is similar to that of the MAC.
The hash value of a message is encrypted with a user’s private key.
Anyone who knows the user’s public key can verify the integrity of the message.
An attacker who wishes to alter the message would need to know the user’s private key.
Implications of digital signatures go beyond just message authentication.

Key Management

2023-11-09T05:56:00.003-08:00

Key Management:-

One of the major roles of public-key encryption has been to address the problem of key distribution. Two distinct aspects to use of public key encryption are present.

The distribution of public keys.

Use of public-key encryption to distribute secret key

Distribution of Public Keys:

The most general schemes for distribution of public keys are given below

Public announcement of public keys
Publicly available directory
Public key authority
Public key certificates

PUBLIC ANNOUNCEMENT OF PUBLIC KEYS:-

Here any participant can send his or her public key to any other participant or broadcast the key to the community at large. For example, many PGP users have adopted the practice of appending their public key to messages that they send to public forums.

Though this approach seems convenient, it has a major drawback. Anyone can forge such a public announcement.

Some user could pretend to be user A and send a public key to another participant or broadcast such a public key. Until the time when A discovers about the forgery and alerts other participants, the forger is able to read all encrypted messages intended for A and can use the forged keys for authentication.

PUBLICLY AVAILABLE DIRECTORY:-

A greater degree of security can be achieved by maintaining a publicly available dynamic directory of public keys. Maintenance and distribution of the public directory would have to be the responsibility of some trusted entity or organization. It includes the following elements:

The authority maintains a directory with a {name, public key} entry for each participant.
Each participant registers a public key with the directory authority. Registration would have to be in person or by some form of secure authenticated communication.
A participant may replace the existing key with a new one at any time, either because of the desire to replace a public key that has already been used for a large amount of data, or because the corresponding private key has been compromised in some way.
Participants could also access the directory electronically. For this purpose, secure, authenticated communication from the authority to the participant is mandatory. This scheme has still got some vulnerabilities. If an adversary succeeds in obtaining or computing the private key of the directory authority, the adversary could authoritatively pass out counterfeit public keys and subsequently impersonate any participant and eavesdrop on messages sent to any participant. Or else, the adversary may tamper with the records kept by the authority.

PUBLIC-KEY AUTHORITY:-

Stronger security for public-key distribution can be achieved by providing tighter control over the distribution of public keys from the directory. This scenario assumes the existence of a public authority (whoever that may be) that maintains a dynamic directory of public keys of all users.

The public authority has its own (private key, public key) that it is using to communicate to users. Each participant reliably knows a public key for the authority, with only the authority knowing the corresponding private key. For example, consider that Alice and Bob wish to communicate with each other and the following steps take place and are also shown in the figure below:

1.) Alice sends a timestamped message to the central authority with a request for Bob’s public key (the time stamp is to mark the moment of the request)

2.) The authority sends back a message encrypted with its private key (for authentication) –message contains Bob’s public key and the original message of Alice – this way Alice knows this is not a reply to an old request;

3.) Alice starts the communication to Bob by sending him an encrypted message containing her identity IDA and a nonce N1 (to identify uniquely this transaction)

4.) Bob requests Alice’s public key in the same way (step 1)

5.) Bob acquires Alice’s public key in the same way as Alice did. (Step-2)

6.) Bob replies to Alice by sending an encrypted message with N1 plus a new generated nonce N2 (to identify uniquely the transaction)

7.) Alice replies once more encrypting Bob’s nonce N2 to assure bob that its correspondent is Alice

Thus, a total of seven messages are required. However, the initial four messages need be used only infrequently because both A and B can save the other's public key for future use, a technique known as caching. Periodically, a user should request fresh copies of the public keys of its correspondents to ensure currency.

PUBLIC-KEY CERTIFICATES:-

The above technique looks attractive, but still has some drawbacks. For any communication between any two users, the central authority must be consulted by both users to get the newest public keys i.e. the central authority must be online 24 hours/day.

If the central authority goes offline, all secure communications get to a halt. This clearly leads to an undesirable bottleneck.

A further improvement is to use certificates, which can be used to exchange keys without contacting a public-key authority, in a way that is as reliable as if the keys were obtained directly from a public-key authority. A certificate binds an identity to public key, with all contents signed by a trusted Public-Key or Certificate Authority (CA).

A user can present his or her public key to the authority in a secure manner, and obtain a certificate. The user can then publish the certificate. Anyone needed this user's public key can obtain the certificate and verify that it is valid by way of the attached trusted signature. A participant can also convey its key information to another by transmitting its certificate.

Other participants can verify that the certificate was created by the authority. This certificate issuing scheme does have the following requirements:

Any participant can read a certificate to determine the name and public key of the certificate's owner.
Any participant can verify that the certificate originated from the certificate authority and is not counterfeit.
Only the certificate authority can create and update certificates.
Any participant can verify the currency of the certificate.

Application must be in person or by some form of secure authenticated communication. For participant A, the authority provides a certificate of the form CA = E(PRauth, [T||IDA||PUa]) where PRauth is the private key used by the authority and T is a timestamp.
A may then pass this certificate on to any other participant, who reads and verifies the certificate as follows: D(PUauth, CA) = D(PUauth, E(PRauth, [T||IDA||PUa])) = (T||IDA||PUa)
The recipient uses the authority's public key, PUauth to decrypt the certificate. Because the certificate is readable only using the authority's public key, this verifies that the certificate came from the certificate authority.
The elements IDA and PUa provide the recipient with the name and public key of the certificate's holder. The timestamp T validates the currency of the certificate.
The timestamp counters the following scenario. A's private key is learned by an adversary. A generates a new private/public key pair and applies to the certificate authority for a new certificate. Meanwhile, the adversary replays the old certificate to B. If B then encrypts messages using the compromised old public key, the adversary can read those messages.
In this context, the compromise of a private key is comparable to the loss of a credit card. The owner cancels the credit card number but is at risk until all possible communicants are aware that the old credit card is obsolete. Thus, the timestamp serves as something like an expiration date. If a certificate is sufficiently old, it is assumed to be expired.
One scheme has become universally accepted for formatting public-key certificates: the

X.509 standard. X.509 certificates are used in most network security applications, including IP security, secure sockets layer (SSL), and secure electronic transactions (SET).

SECRET KEY DISTRIBUTION WITH CONFIDENTIALITY AND AUTHENTICATION:-

It is assumed that A and B have exchanged public keys by one of the schemes described earlier. Then the following steps occur:

1. A uses B's public key to encrypt a message to B containing an identifier of A (IDA) and a nonce (N1), which is used to identify this transaction uniquely.

2. B sends a message to A encrypted with PUa and containing A's nonce (N1) as well as a new nonce generated by B (N2) Because only B could have decrypted message (1), the presence of N1 in message (2) assures A that the correspondent is B.

3. A returns N2 encrypted using B's public key, to assure B that its correspondent is A.

4. A selects a secret key Ks and sends M = E(PUb, E(PRa, Ks)) to B. Encryption of this message with B's public key ensures that only B can read it; encryption with A's private key ensures that only A could have sent it.

5. B computes D(PUa, D(PRb, M)) to recover the secret key.

The result is that this scheme ensures both confidentiality and authentication in the exchange of a secret key.

Asymmetric Cryptography

2023-11-01T10:35:00.002-07:00

Asymmetric key cryptography:-

In asymmetric cryptography, there are two keys: a private key and a public key. The private key is kept by the receiver. The public key is announced to the public.

public key:-

The development of public-key cryptography is the greatest and perhaps the only true revolution in the entire history of cryptography. It is asymmetric, involving the use of two separate keys, in contrast to symmetric encryption, which uses only one key. Public key schemes are neither more nor less secure than private key (security depends on the key size for both). Public-key cryptography complements rather than replaces symmetric cryptography. Both also have issues with key distribution, requiring the use of some suitable protocol.

The concept of public-key cryptography evolved from an attempt to attack two of the most difficult problems associated with symmetric encryption:

1.) key distribution – how to have secure communications in general without having to trust a KDC with your key

2.) Digital signatures – how to verify a message comes intact from the claimed sender

Public-key/two-key/asymmetric cryptography involves the use of two keys:

a public-key, which may be known by anybody, and can be used to encrypt messages, and verify signatures

a private-key, known only to the recipient, used to decrypt messages, and sign (create) signatures.

is asymmetric because those who encrypt messages or verify signatures cannot decrypt messages or create signatures

Public-Key algorithms rely on one key for encryption and a different but related key for decryption. These algorithms have the following important characteristics:

It is computationally infeasible to find decryption key knowing only algorithm & encryption key
It is computationally easy to en/decrypt messages when the relevant (en/decrypt) key is known

The following figure illustrates public-key encryption process and shows that a public-key encryption scheme has six ingredients: plaintext, encryption algorithm, public & private keys, ciphertext& decryption algorithm.

The essential steps involved in a public-key encryption scheme are given below:

1) Each user generates a pair of keys to be used for encryption and decryption.

2) Each user places one of the two keys in a public register and the other key is kept private.

3) If B wants to send a confidential message to A, B encrypts the message using A‟s public key.

4) When A receives the message, she decrypts it using her private key. Nobody else can decrypt the

message because that can only be done using A‟s private key (Deducing a private key should be infeasible).

5) If a user wishes to change his keys –generate another pair of keys and publish the public one: no interaction with other users is needed.

The first attack on Public-key Cryptography is the attack on Authenticity. An attacker may impersonate user B: he sends a message and claims in the message to be B –A has no guarantee this is so. To overcome this, B will encrypt the message using his private key. Receiver decrypts using B’s public key. This shows the authenticity of the sender because (supposedly) he is the only one who knows the private key. The entire encrypted message serves as a digital signature. This scheme is depicted in the following figure:

Symmetric cryptography

2023-10-28T10:12:00.003-07:00

Cryptography:-

We can divided all the cryptography algorithms(ciphers) into three groups:

Symmetric cryptography
Asymmetric cryptography
Hash Function

Symmetric cryptography:-

In symmetric-key cryptography, the same key is used by both parties. The sender uses this key and an encryption algorithm to encrypt data; the receiver uses the same key and the corresponding decryption algorithm to decrypt the data.

A symmetric encryption scheme has five ingredients:

• Plaintext: This is the original intelligible message or data that is fed into the algorithm as input.

• Encryption algorithm: The encryption algorithm performs various substitutions and transformations on the plaintext.

• Secret key: The secret key is also input to the encryption algorithm. The key is a value independent of the plaintext and of the algorithm.

• Ciphertext: This is the scrambled message produced as output. It depends on the plaintext and the secret key. For a given message, two different keys will produce two different ciphertexts. The ciphertext is an apparently random stream of data and, as it stands, is unintelligible.

• Decryption algorithm: This is essentially the encryption algorithm run in reverse. It takes the ciphertext and the secret key and produces the original plaintext.

Private key Cryptography:-

Symmetric encryption also called private-key encryption or secret-key encryption involves using the same key for encryption and decryption.

Model for Network Security

2023-10-28T07:02:00.005-07:00

Network Security Model:-

A message is to be transferred from one party to another across some sort of Internet service. The two parties, who are the principals in this transaction, must cooperate for the exchange to take place. A logical information channel is established by defining a route through the Internet from source to destination and by the cooperative use of communication protocols (e.g., TCP/IP) by the two principals.

Security aspects come into play when it is necessary or desirable to protect the information transmission from an opponent who may present a threat to confidentiality, authenticity, and so on. All the techniques for providing security have two components:

• A security-related transformation on the information to be sent. Examples include the encryption of the message, which scrambles the message so that it is unreadable by the opponent, and the addition of a code based on the contents of the message, which can be used to verify the identity of the sender.

• Some secret information shared by the two principals and, it is hoped, unknown to the opponent. An example is an encryption key used in conjunction with the transformation to scramble the message before transmission and unscramble it on reception.

A trusted third party may be needed to achieve secure transmission. For example, a third party may be responsible for distributing the secret information to the two principals while keeping it from any opponent. Or a third party may be needed to arbitrate disputes between the two principals concerning the authenticity of a message transmission.

This general model shows that there are four basic tasks in designing a particular security service:

1. Design an algorithm for performing the security-related transformation. The algorithm should be such that an opponent cannot defeat its purpose.

2. Generate the secret information(keys) to be used with the algorithm.

3. Develop methods for the distribution and sharing of the secret information.

4. Specify a protocol to be used by the two principals that makes use of the security algorithm and the secret information to achieve a particular security service.

using this model requires us to:

select appropriate gatekeeper functions to identify users
implement security controls to ensure only authorized users access designated information or resources

Security Attacks

2023-10-28T01:56:00.002-07:00

X.800 Security Attacks:-

There are two types of Software attacks:

Active Attacks
Passive Attacks

Active Attacks:-

An active attack attempts to alter system resources or affect their operation. Active attacks Involve some modification of the data stream or the creation of a false stream.

Active attacks are difficult to prevent because of the wide variety of potential physical, software, and network vulnerabilities. Goal is to detect attacks and to recover from any disruption or delays caused by them

• Active attacks can be subdivided into four categories:

a) Masquerade.

b) Replay.

c) Modification of messages.

d) Denial of service.

Masquerade:-

A masquerade takes place when one entity pretends to be a different entity. A masquerade attack usually includes one of the other forms of active attack. For example, authentication sequences can be captured and replayed after a valid authentication sequence has taken place, thus enabling an authorized entity with few privileges to obtain extra privileges by impersonating an entity that has those privileges.

Replay:-

Replay involves the passive capture of a data unit and its subsequent retransmission to produce an unauthorized effect.

Modification of messages:-

Modification of messages simply means that some portion of a legitimate message is altered, or that messages are delayed or reordered, to produce an unauthorized effect. For example, a message meaning “Allow John Smith to read confidential file accounts” is modified to mean “Allow Fred Brown to read confidential file accounts.”

Denial of service:-

The denial of service prevents or inhibits the normal use or management of communications facilities. This attack may have a specific target; for example, an entity may suppress all messages directed to a particular destination (e.g., the security audit service).Another form of service denial is the disruption of an entire network, either by disabling the network or by overloading it with messages so as to degrade performance.

Passive Attacks:-

A passive attack attempts to learn or make use of information from the system but does not affect system resources.

Passive attacks are in the nature of eavesdropping on, or monitoring of, transmissions. The goal of the opponent is to obtain information that is being transmitted.

Two types of passive attacks are:

• a) Release of message contents.

• b) Traffic analysis.

Release of message contents:-

The release of message contents is easily understood. A telephone conversation, an electronic mail message, and a transferred file may contain sensitive or confidential information. We would like to prevent an opponent from learning the contents of these transmissions.

Traffic analysis:-

A second type of passive attack, traffic analysis, is subtler. Suppose that we had a way of masking the contents of messages or other information traffic so that opponents, even if they captured the message, could not extract the information from the message. The common technique for masking contents is encryption. If we had encryption protection in place, an opponent might still be able to observe the pattern of these messages. The opponent could determine the location and identity of communicating hosts and could observe the frequency and length of messages being exchanged. This information might be useful in guessing the nature of the communication that was taking place.

Active Attacks	Passive Attacks
Data modification	Data monitoring
Affect the System	Does not affect the system
Easy to detect	Difficult to detect
Attacks on availability and integrity	Attacks on confidentiality
Examples: Masquerade Replay Denial of services Modification of messages	Examples: Release of message content Traffic Analysis
Techniques to avoid: Firewalls IPS(Intrusion Prevention System)	Techniques to avoid: Encryption

Security Mechanisms:-

2023-10-22T07:25:00.001-07:00

Security Mechanisms (X.800):-

Security mechanisms are used to implement security services. Lists of security mechanisms defined in X.800. The mechanisms are divided into those that are implemented in a specific protocol layer, such as TCP or an application-layer protocol, and those that are not specific to any particular protocol layer or security service.

Specific security mechanisms

Encipherment
Digital signatures
Access controls
Data integrity
Authentication exchange
traffic padding
routing control
notarization

Pervasive security mechanisms

Trusted functionality
Security labels
Event detection
Security audit trails
Security recovery

• Encipherment:-

The use of mathematical algorithms to transform data into a form that is not readily intelligible. The transformation and subsequent recovery of the data depend on an algorithm and zero or more encryption keys.

• Digital signature:-

Data appended to, or a cryptographic transformation of, a data unit that allows a recipient of the data unit to prove the source and integrity of the data unit and protect against forgery (e.g., by the recipient).

• Access Control:-

A variety of mechanisms that enforce access rights to Resources.

• Data Integrity:-

A variety of mechanisms used to assure the integrity of a data unit or stream of data units.

• Authentication Exchange:-

A mechanism intended to ensure the identity of an entity by means of information exchange.

• Traffic Padding:-

The insertion of bits into gaps in a data stream to frustrate eavesdropper’s traffic analysis attempts.

• Routing Control:-

Enables selection of particular physically secure Routes for certain data and allows routing changes, especially when a breach of security is suspected.

• Notarization:-

The use of a trusted third party to assure certain properties of a data exchange.

• Trusted Functionality:-

That which is perceived to be correct with respect to some criteria(e.g., as establish by a security policy).

• Security labels:-

The marking bound to a resource (which may be a data unit) that name or designates the security attributes of that resources.

• Event Detection:-

Detection of security-relevant events.

• Security Audit Trails:-

Data collected and potentially used to facilitate a security audit, which is an independent review and examination of system records and activities.

• Security recovery:-

Deal with requests from mechanisms, such as event handling and management functions and takes recovery functions.

Security Services

2023-10-22T05:34:00.006-07:00

Security Services (X.800):-

Requirements X.800 defines security services in following categories.

Authentication
Access control
Data confidentiality
Data integrity
Nonrepudiation
Availability services

1. Authentication:-

The authentication service is concerned with assuring that a communication is authentic:

The recipient of the message should be sure that the message came from the source that it claims to be

All communicating parties should be sure that the connection is not interfered with by unauthorized party.

Example: consider a person, using online banking service. Both the user and the bank should be assured in identities of each other.

2. Access control:-

The prevention of unauthorized use of a resource (i.e., this service controls who can have access to a

resource, under what conditions access can occur, and what those accessing the resource are allowed to

do).

3. Data confidentiality:-

Protection of data from unauthorized disclosure. It includes:

Connection confidentiality
Connectionless confidentiality
Selective field confidentiality
Traffic-Flow Confidentiality

4. Data Integrity:-

The assurance that data received are exactly as sent by an authorized entity (i.e., contain no modification, insertion, deletion, or replay).

5. Nonrepudiation:-

Provides protection against denial by one of the entities involved in a communication of having

participated in all or part of the communication.

Nonrepudiation can be related to

Origin: proof that the message was sent by the specified party
Destination: proof that the message was received by the specified party

Example: Imagine a user of online banking who has made a transaction, but later denied that. How the

bank can protect itself in such situation?

6. Availability service:-

The property of a system or a system resource being accessible and usable upon demand by an authorized system entity, according to performance specifications for the system (i.e., a system is available if it provides services according to the system design whenever users request them).

OSI Security Architecture

2023-10-21T23:59:00.001-07:00

ITU-T X.800 Security Architecture for OSI:-

The OSI Security Architecture is a framework that provides a systematic way

of defining the requirements for security and characterizing the approaches to satisfying those

requirements. The OSI (Open Systems Interconnection) Security Architecture defines a systematic

approach to providing security at each layer. It defines security services and security mechanisms that

can be used at each of the seven layers of the OSI model to provide security for data transmitted over a

network. These security services and mechanisms help to ensure the confidentiality, integrity, and

availability of the data.

The document defines security attacks, mechanisms, and services, and the relationships among these

Categories.

ITU-T: International Telecommunication Union Telecommunication Standardization Sector

OSI: Open Systems Interconnections

• Security attack:-

Any action that compromises the security of information owned by an organization.

• Security mechanism:-

A process (or a device incorporating such a process) that is designed to detect, prevent, or

recover from a security attack.

• Security service:-

A processing or communication service that enhances the security of the data processing

systems and the information transfers of an organization. The services are intended to counter security

attacks, and they make use of one or more security mechanisms to provide the service.

Principle security

2023-10-21T23:30:00.001-07:00

There are five principles of security. They are as follows:

Confidentiality:-

The principle of confidentiality specifies that only the sender and the intended recipient should be able to access the content of the message.

Integrity:-

The confidential information sent by A to B which is accessed by C without the permission or knowledge of A and B.

Authentication:-

Authentication mechanism helps in establishing proof of identification.

Access control:-

Access control specifies and control who can access what.

Availability:-

It means that assets are accessible to authorized parties at appropriate times.

Database system, Information security, Math Online Exercises, Past Papers - Class 11 and Class 12,

Decision tree

ID3:

Multiple Linear Regressions

code:

Output:

Simple linear regression

Code:

Clustering and Association

Clustering

Association:

Classification and Regression

Classification

Example

Examples

Algorithms

Remarks

Regression

Example

General approach

Example

regression Algorithms

Naive Bayes classifier

Types of Machine Learning

Machine Learning Model

Machine Learning Model:

Understanding data

1. Different forms of data

Learning

Definition of learning:

Introduction to machine Learning

Machine Learning

Introduction to Machine LearningLearningUnderstanding dataMachine Learning ModelTypes of Machine LearningClassification and RegressionClustering and AssociationSimple linear regressionMultiple Linear RegressionNaive Bayes classifier Decision Tree

EXPLORING COMPUTER AND THEIR USES

OVERVIEW:

THE COMPUTER DEFINED:

COMPUTER FOR INDIVIDUAL USERS:

DESKTOP COMPUTERS:

WORKSTATIONS:

NOTEBOOK COMPUTERS:

TABLET PCs:

HANDHELD COMPUTERS:

SMARTPHONES:

COMPUTERS FOR ORGANIZATION:

NETWORK SERVERS:

MAINFRAME COMPUTERS:

MINICOMPUTERS:

SUPERCOMPUTERS:

COMPUTERS IN SOCIETY:

HOME:

EDUCATION:

INDUSTRY:

GOVERNMENT:

Application of Information & Communication Technology

Encryption Algorithms

Encryption Algorithms:-

Data Encryption Standard (DES):-

History:-

working:-

AES (ADVANCED ENCRYPTION ALGORITHM):-

Substitute Bytes:-

Shift Row Transformation:-

MIX COLUMN TRANSFORMATION:-

ADD ROUND KEY TRANSFORMATION:-

Digital Signature

Digital signature:-

Key Management

Key Management:-

Distribution of Public Keys:

PUBLIC ANNOUNCEMENT OF PUBLIC KEYS:-

PUBLICLY AVAILABLE DIRECTORY:-

PUBLIC-KEY AUTHORITY:-

PUBLIC-KEY CERTIFICATES:-

SECRET KEY DISTRIBUTION WITH CONFIDENTIALITY AND AUTHENTICATION:-

Asymmetric Cryptography

Asymmetric key cryptography:-

public key:-

Symmetric cryptography

Cryptography:-

Symmetric cryptography:-

Introduction to Machine Learning
Learning
Understanding data
Machine Learning Model
Types of Machine Learning
Classification and Regression
Clustering and Association
Simple linear regression
Multiple Linear Regression
Naive Bayes classifier
Decision Tree