Archive for March 26th, 2022

Hc fin apply

For this assignment, each students situation is different; therefore, there is no right or wrong answer.  The key to this assignment is for you to:

  • Predict what course topics/objectives that you will utilize in your career.
  • Explain these course topics/objectives to demonstrate your understanding.
  • Decide (and explain) how you will adapt these to meet your career needs.

Scoring will be based on these three key aspects.  This assignment is a self-analysis of the course to your career objectives.  Please create this paper in APA formatting with 2-3 pages of detail information.  These 2-3 pages do not include the title page and the reference page

CMT219-Coursework-202122v4.pdf

Page 1 of 9

Cardiff School of Computer Science and Informatics Coursework Assessment Pro-forma

Module Code: CMT219 Module Title: Algorithms, Data Structures and Programming Lecturer: Dr Daniela Tsaneva, Neetesh Saxena Assessment Title: Algorithms Data Structures and Design Patterns Assessment Number: 2 Date Set: 25th March 2022 Submission Date & Time: 13th May 2022 at 9:30am British Summer Time Return Date: 10th June 2022

This coursework is worth 50% of the total marks available for this module. If coursework is submitted late (and where there are no extenuating circumstances):

1. If the assessment is submitted no later than 24 hours after the deadline, the mark for the assessment will be capped at the minimum pass mark;

2. If the assessment is submitted more than 24 hours after the deadline, a mark of 0 will be given for the assessment.

Your submission must include the official Coursework Submission Cover sheet, which can be found here: https://docs.cs.cf.ac.uk/downloads/coursework/Coversheet.pdf

Submission Instructions

Your coursework should be submitted through Learning Central. Your submission should include:

Description Type Name

Cover sheet Compulsory PDF (.pdf) file [student number].pdf

Part

1

ONE ZIP archive file (and no more than one) containing all the source code files for your answer to questions of Part 1

Compulsory One ZIP (.zip) archive

[student number]_Part1Code.zip

e.g., C1234567_Part1Code.zip

ONE PDF file (and no more than one) which contains a written justification for your design of the program and screen shots showing an example of the output of each application

Compulsory One PDF (.pdf) file

[student number]_Part1.pdf

e.g., C1234567_ Part1.pdf

Part

2

ONE PDF file (and no more than one) which contains the diagrams, descriptions, explanations, discussion, program listings (extracts from the code which show clearly what you did in order to complete the Tasks).

Compulsory PDF (.pdf) file [student number]_Part2.pdf e.g., C1234567_ Part2.pdf

ONE ZIP archive file (and no more than one) containing all the source code files for your answers for Part 2

Compulsory ZIP (.zip) archive file

[student number]_Part2Code.zip e.g., C1234567_Part2Code.zip

Any code submitted will be run on a university machine and must be submitted as stipulated in the instructions above.

Any deviation from the submission instructions above (including the number and types of files submitted) will result in a reduction of 20% of the attained mark.

Staff reserve the right to invite students to a meeting to discuss coursework

submissions

Page 2 of 9

Assignment

PART 1 – Algorithms and Data Structure

[Total: 50 marks]

All code must be written by you, although you can use the lecture notes (and lab exercises), textbooks, and the Oracle Java website for guidance.

A) Extracting words from a text document is the first step for many text-based

applications of artificial intelligence, e.g., detecting abusive tweets on Twitter. This task asks you to extract all valid words from the document “Input219.txt” based on a given vocabulary “google-10000-english-no-swears.txt1” (you can download both files from the Learning Central). Specifically, if a word token from “Input219.txt” matches a word in “google-10000-english-no-swears.txt” (case insensitively), you keep that word, otherwise you discard it. You should use ArrayList to store data within the Java program, so the return of this task is an ArrayList containing all valid words from “Input219.txt”. Note that these two text files are relatively big, you should consider how to make your program efficient.

[30 marks]

(Functionality: 16, Design: 10, Ease of use: 2, Presentation: 2)

B) Implement the merge sort (the pseudocode for merge sort is available in the lecture slides) in order to sort the words obtained in question (A) in alphabetical order, i.e., the output of your program will be the sorted words in alphabetical order. For the merge sort algorithm write a method e.g. mergeSort(…), measure

• time that is needed to sort the first 100 of the words, first 200 of the words, first 300 of the words, etc. up to all words by the algorithm.

• count the moves/comparisons that occur while sorting elements

(Before attempting this exercise you should work through the Algorithms lab exercises, available on Learning Central. The techniques used there will help you to work out how to approach this part of the coursework, in particular there are examples on how to time algorithms and count the moves and swaps.)

[20 marks] (Functionality: 12, Design: 4, Ease of use: 2, Presentation: 2)

1 This vocabulary contains a list of the 10,000 most common English words in order of frequency of the Google's Trillion Word Corpus.

Page 3 of 7

Criteria for Assessment of PART 1

Credit will be awarded against the following criteria.

Functionality • To what extent does the program perform the task(s) required by the question

and how efficient it is. Design

• How well designed is the code, particularly with respect to the ease with which it may be maintained or extended. In particular, consideration will be given to: • Use of appropriate types, program control structures and classes from the

core API. • Usage and structure of appropriate classes and methods. • Consideration of achieving optimised algorithms. • An ‘excellent’ design of code will carefully consider using most suitable data

structures and aiming for efficient and elegant algorithms. A ‘bad’ design of code pays little/no attention to data structure and algorithm efficiency.

Ease of use

• Formatting of input/output. • Interaction with the user. • How does the code deal with invalid user input? Will the applications crash for

certain data?

Documentation and presentation • Appropriate use of comments. • Readability of code. • Clear and appropriate screenshots.

Page 4 of 7

Criteria for Assessment of PART 1

Criteria Distinction (70-100%)

Merit (60-69%)

Pass (50-59%)

Fail (0-49%)

PART 1 – Algorithms and Data Structure

Functionality Fully working application that demonstrates an excellent understanding of the assignment problem using relevant Java approach.

All required functionality is met, and the application is working properly with some minors’ errors

Some of the functionality developed with and incorrect output major errors.

Faulty application with wrong implementation and wrong output.

Design Using most suitable data structures and efficient and elegant algorithms.

Using good data structures and suitable algorithms.

Some of the data structures are not suitable and the algorithms not efficient.

Pays little/no attention to data structure and algorithm efficiency.

Ease of use Excellent formatting of input/output, and interaction with the user, the application deals well with invalid user input and doesn’t crash for any data.

Good formatting of input/output, and interaction with the user, the application has some minor errors with invalid user input and doesn’t crash for most data.

Suitable formatting of input/output, and interaction with the user, the application doesn’t deal with invalid user input and crashes for some data.

Bad formatting of input/output, and unsuitable interaction with the user, the application doesn’t deal with invalid user input and crashes for most data.

Documentation and Presentation

Appropriate use of comments, readability of code, clear and appropriate screenshots.

Some use of comments, the code is mostly readable, reasonable screenshots.

Few comments, not good readability of code, not clear screenshots.

No comments, bad readability of code, no screenshots.

Page 5 of 7

PART 2 – Design and Implementation of Design Patterns

[Total: 50 marks]

Note: A list of files is included in the present coursework specification at the end for your convenience.

Download the ZIP file from Learning Central that contains the source code files of this task: Product.java, ProductRecommender.java, ChoiceStrategy.java, MostFuturisticStrategy.java, MostPracticalStrategy.java. This Java program creates two new products and then chooses between them, first by applying the MostFuturisticStrategy and then the MostPracticalStrategy. Futuristicness and practicality are both rated on a scale between 1(low) and 5 (high).

A) State the purpose of the Strategy design pattern. [4 marks]

B) Complete the implementation of the MostFuturisticStrategy class with a

chooseBetween (Product a, Product b) method that returns Product a if its

futuristicness is greater than or equal to the futuristicness of Product b; and

returns Product b otherwise. [8 marks]

C) Complete the implementation of MostPracticalStrategy with a chooseBetween

(Product a, Product b) method that returns Product a if its practicality is greater

than or equal to the practicality of Product b; and returns

Product b otherwise. [8 marks]

D) Modify the doExample () method in the ProductRecommender class, so that

each of the strategies can be applied, in order generate the following output:

Current strategy: choose most futuristic Chosen vehicle: DeLorean DMC-12 Strategy changed: choose most practical

Chosen vehicle: LDV Maxus [10 marks]

E) Draw a UML class diagram representing the resulting Strategy design pattern in

this program. [10 marks]

F) Explain in your own words the role of each of the classes and the changes made

in realising this design pattern as well the relationships between these classes

and the interface. [10 marks]

Criteria for Assessment of PART 2

Credit will be awarded against the following specific criteria. Ability to:

• Understand the purpose of a Java program and apply principles of good object-

oriented design in order to create a critique and/or modify the supplied programs

(check the source code files provided);

• Apply and implement design patterns that pertain to a specified programming task;

• Reflect on the application of design patterns in OO Java programs.

Page 6 of 7

Feedback on your performance will address each of the above criteria.

Criteria for Assessment of PART 2

Distinction (70-100%)

Merit (60-69%)

Pass (50-59%)

Fail (0-49%)

PART 2 – Design and Implementation of Design Patterns

An excellent implementation, showing a good degree of insight and reflection, and the code is of good quality.

The tasks have been completed, the code “works”, and some clear insight has been shown.

The tasks have been completed, the code “works”, but minimal insight has been shown.

The submitted answer only addresses the tasks to a very limited extent. Some attempt has been made to address the tasks, and the code mostly works.

Learning Outcomes Assessed This assignment particularly addresses the following module learning outcomes:

• Critically evaluate design patterns and their applicability to different scenarios

• Select and use appropriate algorithms and data structures to provide best

performance in a given scenario

Code Reuse Your solutions may make use of any classes in the Core Java API. You may also reproduce small pieces of code from:

• The CMT219 course handouts and solutions

• java.oracle.com

• any textbooks provided:

• The section reproduced does not form the entire solution to a single question

• The source of the code is clearly referenced in your source code

• Your code is commented to demonstrate clearly that you understand how the

reproduced code works (i.e., explain why types have been selected, why other

language features have been used, etc.)

You may NOT reproduce code written by any other student or code downloaded from any other website.

If you are in any doubt about whether you may include a piece of code that you have not written yourself, ask the lecturer before submitting.

See “Referencing in code guidance” at Learning Central → COMSC-SCHOOL → Learning Materials → Referencing in code guidance.

Feedback and Suggestion for Future Learning Feedback on your coursework will address the above criteria. Feedback and marks will be returned via Learning Central. There will be opportunity for individual feedback during an agreed time.

Page 7 of 7

Appendix: Program listing for PART 2

public class ProductRecommender { ChoiceStrategy

myStrategy;

public static void main(String args[]) {

ProductRecommender recommender=new ProductRecommender();

recommender.doExample();

}

public void doExample() {

Product p1=new Product("DeLorean DMC-12", 5, 1);

Product p2=new Product("LDV Maxus", 1, 5);

System.out.println("Current strategy: choose most futuristic");

// Add code here to create a MostFuturisticStrategy and

// print out the chosen vehicle according to this strategy

System.out.println("Strategy changed: choose most practical");

// Add code here to create a MostPracticalStrategy and

// print out the chosen vehicle according to this strategy

}

}

public interface ChoiceStrategy { public Product

chooseBetween(Product a, Product b);

}

public class MostFuturisticStrategy implements ChoiceStrategy {

// Complete this with an implementation of

// chooseBetween(Product a, Product b) which returns

// Product a if its futuristicness is greater than or equal

// to that of b; and returns Product b otherwise

}

public class MostPracticalStrategy implements ChoiceStrategy {

// Complete this with an implementation of

// chooseBetween(Product a, Product b) which returns

// Product a if its practicality is greater than or equal

// to that of b; and returns Product b otherwise

}

public class Product { public String

name; public int

futuristicness;

public int practicality;

public Product(String theName, int theFuturisticness, int thePracticality) { name=theName;

futuristicness=theFuturisticness;

practicality=thePracticality;

}

}

Code-part2.zip

Code-part4/Product.java

Code-part4/Product.java

public class Product {
  public String name;
  public int futuristicness;
  public int practicality;

  public Product(String theName, int theFuturisticness, int thePracticality) {
    name=theName; futuristicness=theFuturisticness; practicality=thePracticality;
  }
}

__MACOSX/Code-part4/._Product.java

Code-part4/ChoiceStrategy.java

Code-part4/ChoiceStrategy.java

public interface ChoiceStrategy {
  public Product chooseBetween(Product a, Product b);
}

__MACOSX/Code-part4/._ChoiceStrategy.java

Code-part4/MostFuturisticStrategy.java

Code-part4/MostFuturisticStrategy.java

public class MostFuturisticStrategy implements ChoiceStrategy {
     // Complete this with an implementation of
    // chooseBetween(Product a, Product b) which returns
    // Product a if its futuristicness is greater than or equal
    // to that of b; and returns Product b otherwise
}


__MACOSX/Code-part4/._MostFuturisticStrategy.java

Code-part4/ProductRecommender.java

Code-part4/ProductRecommender.java

public class ProductRecommender {
  ChoiceStrategy myStrategy;

  public static void main(String args[]) {
    ProductRecommender recommender=new ProductRecommender();
    recommender.doExample();
  }

  public void doExample() {
    Product p1=new Product("DeLorean DMC-12", 5, 1);
    Product p2=new Product("LDV Maxus", 1, 5);

     System.out.println("Current strategy: choose most futuristic");
     // Add code here to create a MostFuturisticStrategy and
     // print out the chosen vehicle according to this strategy

        System.out.println("Strategy changed: choose most practical");
    // Add code here to create a MostPracticalStrategy and
    // print out the chosen vehicle according to this strategy

  }
}

__MACOSX/Code-part4/._ProductRecommender.java

Code-part4/MostPracticalStrategy.java

Code-part4/MostPracticalStrategy.java

public class MostPracticalStrategy implements ChoiceStrategy {
    // Complete this with an implementation of
    // chooseBetween(Product a, Product b) which returns
    // Product a if its practicality is greater than or equal
    // to that of b; and returns Product b otherwise
}
   

__MACOSX/Code-part4/._MostPracticalStrategy.java

Research paper

 

  1. This week, you have read about entrepreneurship in a global economy. For your written assignment this week, complete a case study of the organization you work for (use a hypothetical or other organization if more applicable) that will address the following prompts:
    • Describe the organizations environment, and evaluate its preparedness to go global, if not already, and it’s strategy for staying global if it is.
    • Research other company’s strategy for going global and explain if this will or will not work for your company. 
    • Make a recommendation for a global strategy in the organization, including a justification for your recommendations.
    • Submit your midterm research paper as a single document. Your paper should meet the following requirements:
    • Be approximately four to six pages in length, not including the required cover page and reference page.
    • Follow APA 7 guidelines. Your paper should include an introduction, a body with fully developed content, and a conclusion.
    • Support your answers with the readings from the course and at least two scholarly journal articles to support your positions, claims, and observations, in addition to your textbook. The school Library is a great place to find resources.
    • Be clearly and well-written, concise, and logical, using excellent grammar and style techniques. You are being graded in part on the quality of your writing.

3/1 Devote

During the COVID-19 stay-at-home requirements, many people worked from home. To carry on business, companies utilized various web-based meeting tools (such as Zoom). Some of these tools lacked robust security. A company, being proactive, sought to assure security against those attempting to take over such sessions.

In Christianity, God brings to mind those things we need to ensure protection against, including issues related to computer security. We again look to the word of God for Scripture that tells us that safety can be found in the Lord.  says, Keep me safe, Lord, from the hands of the wicked; protect me from the violent, who devise ways to trip my feet (NIV). We find in the Bible the admonition to seek the Lord in our activities as protection from those that would trick us.

  1. Review the material in the Getting Started section, including the specific Bible passage.
  2. Be sure to address to the following prompts in your paper:
    1. How do you see this verse applying to the attackers attempts at social engineering?
    2. How would you apply the principles of these verses in your workplace?
  3. Your paper should be at least 250 words in length.
  4. Submit your assignment by the end of the workshop.

ITS632_Chapter7.ppt

Data Mining
Cluster Analysis: Basic Concepts
and Algorithms

Lecture Notes for Chapter 7

Introduction to Data Mining

by

Tan, Steinbach, Kumar

© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 *

What is Cluster Analysis?

  • Finding groups of objects such that the objects in a group will be similar (or related) to one another and different from (or unrelated to) the objects in other groups

Inter-cluster distances are maximized

Intra-cluster distances are minimized

Applications of Cluster Analysis

  • Understanding
  • Group related documents for browsing, group genes and proteins that have similar functionality, or group stocks with similar price fluctuations
  • Summarization
  • Reduce the size of large data sets

What is not Cluster Analysis?

  • Supervised classification
  • Have class label information
  • Simple segmentation
  • Dividing students into different registration groups alphabetically, by last name
  • Results of a query
  • Groupings are a result of an external specification
  • Graph partitioning
  • Some mutual relevance and synergy, but areas are not identical

Types of Clusterings

  • A clustering is a set of clusters
  • Important distinction between hierarchical and partitional sets of clusters
  • Partitional Clustering
  • A division data objects into non-overlapping subsets (clusters) such that each data object is in exactly one subset
  • Hierarchical clustering
  • A set of nested clusters organized as a hierarchical tree

Other Distinctions Between Sets of Clusters

  • Exclusive versus non-exclusive
  • In non-exclusive clusterings, points may belong to multiple clusters.
  • Can represent multiple classes or ‘border’ points
  • Fuzzy versus non-fuzzy
  • In fuzzy clustering, a point belongs to every cluster with some weight between 0 and 1
  • Weights must sum to 1
  • Probabilistic clustering has similar characteristics
  • Partial versus complete
  • In some cases, we only want to cluster some of the data
  • Heterogeneous versus homogeneous
  • Cluster of widely different sizes, shapes, and densities

Types of Clusters

  • Well-separated clusters
  • Center-based clusters
  • Contiguous clusters
  • Density-based clusters
  • Property or Conceptual
  • Described by an Objective Function

Types of Clusters: Objective Function

  • Clusters Defined by an Objective Function
  • Finds clusters that minimize or maximize an objective function.
  • Enumerate all possible ways of dividing the points into clusters and evaluate the `goodness' of each potential set of clusters by using the given objective function. (NP Hard)
  • Can have global or local objectives.
  • Hierarchical clustering algorithms typically have local objectives
  • Partitional algorithms typically have global objectives
  • A variation of the global objective function approach is to fit the data to a parameterized model.
  • Parameters for the model are determined from the data.
  • Mixture models assume that the data is a ‘mixture' of a number of statistical distributions.

Types of Clusters: Objective Function …

  • Map the clustering problem to a different domain and solve a related problem in that domain
  • Proximity matrix defines a weighted graph, where the nodes are the points being clustered, and the weighted edges represent the proximities between points
  • Clustering is equivalent to breaking the graph into connected components, one for each cluster.
  • Want to minimize the edge weight between clusters and maximize the edge weight within clusters

Characteristics of the Input Data Are Important

  • Type of proximity or density measure
  • This is a derived measure, but central to clustering
  • Sparseness
  • Dictates type of similarity
  • Adds to efficiency
  • Attribute type
  • Dictates type of similarity
  • Type of Data
  • Dictates type of similarity
  • Other characteristics, e.g., autocorrelation
  • Dimensionality
  • Noise and Outliers
  • Type of Distribution

Clustering Algorithms

  • K-means and its variants
  • Hierarchical clustering
  • Density-based clustering

K-means Clustering

  • Partitional clustering approach
  • Each cluster is associated with a centroid (center point)
  • Each point is assigned to the cluster with the closest centroid
  • Number of clusters, K, must be specified
  • The basic algorithm is very simple

K-means Clustering – Details

  • Initial centroids are often chosen randomly.
  • Clusters produced vary from one run to another.
  • The centroid is (typically) the mean of the points in the cluster.
  • ‘Closeness’ is measured by Euclidean distance, cosine similarity, correlation, etc.
  • K-means will converge for common similarity measures mentioned above.
  • Most of the convergence happens in the first few iterations.
  • Often the stopping condition is changed to ‘Until relatively few points change clusters’
  • Complexity is O( n * K * I * d )
  • n = number of points, K = number of clusters,
    I = number of iterations, d = number of attributes

Evaluating K-means Clusters

  • Most common measure is Sum of Squared Error (SSE)
  • For each point, the error is the distance to the nearest cluster
  • To get SSE, we square these errors and sum them.
  • x is a data point in cluster Ci and mi is the representative point for cluster Ci
  • can show that mi corresponds to the center (mean) of the cluster
  • Given two clusters, we can choose the one with the smallest error
  • One easy way to reduce SSE is to increase K, the number of clusters
  • A good clustering with smaller K can have a lower SSE than a poor clustering with higher K

Problems with Selecting Initial Points

  • If there are K ‘real’ clusters then the chance of selecting one centroid from each cluster is small.
  • Chance is relatively small when K is large
  • If clusters are the same size, n, then

  • For example, if K = 10, then probability = 10!/1010 = 0.00036
  • Sometimes the initial centroids will readjust themselves in ‘right’ way, and sometimes they don’t
  • Consider an example of five pairs of clusters

Solutions to Initial Centroids Problem

  • Multiple runs
  • Helps, but probability is not on your side
  • Sample and use hierarchical clustering to determine initial centroids
  • Select more than k initial centroids and then select among these initial centroids
  • Select most widely separated
  • Postprocessing
  • Bisecting K-means
  • Not as susceptible to initialization issues

Handling Empty Clusters

  • Basic K-means algorithm can yield empty clusters
  • Several strategies
  • Choose the point that contributes most to SSE
  • Choose a point from the cluster with the highest SSE
  • If there are several empty clusters, the above can be repeated several times.

Updating Centers Incrementally

  • In the basic K-means algorithm, centroids are updated after all points are assigned to a centroid
  • An alternative is to update the centroids after each assignment (incremental approach)
  • Each assignment updates zero or two centroids
  • More expensive
  • Introduces an order dependency
  • Never get an empty cluster
  • Can use “weights” to change the impact

Pre-processing and Post-processing

  • Pre-processing
  • Normalize the data
  • Eliminate outliers
  • Post-processing
  • Eliminate small clusters that may represent outliers
  • Split ‘loose’ clusters, i.e., clusters with relatively high SSE
  • Merge clusters that are ‘close’ and that have relatively low SSE
  • Can use these steps during the clustering process
  • ISODATA

Bisecting K-means

  • Bisecting K-means algorithm
  • Variant of K-means that can produce a partitional or a hierarchical clustering

Limitations of K-means

  • K-means has problems when clusters are of differing
  • Sizes
  • Densities
  • Non-globular shapes
  • K-means has problems when the data contains outliers.

Strengths of Hierarchical Clustering

  • Do not have to assume any particular number of clusters
  • Any desired number of clusters can be obtained by ‘cutting’ the dendogram at the proper level
  • They may correspond to meaningful taxonomies
  • Example in biological sciences (e.g., animal kingdom, phylogeny reconstruction, …)

Hierarchical Clustering

  • Two main types of hierarchical clustering
  • Agglomerative:
  • Start with the points as individual clusters
  • At each step, merge the closest pair of clusters until only one cluster (or k clusters) left
  • Divisive:
  • Start with one, all-inclusive cluster
  • At each step, split a cluster until each cluster contains a point (or there are k clusters)
  • Traditional hierarchical algorithms use a similarity or distance matrix
  • Merge or split one cluster at a time

Agglomerative Clustering Algorithm

  • More popular hierarchical clustering technique
  • Basic algorithm is straightforward

Compute the proximity matrix

Let each data point be a cluster

Repeat

Merge the two closest clusters

Update the proximity matrix

Until only a single cluster remains

  • Key operation is the computation of the proximity of two clusters
  • Different approaches to defining the distance between clusters distinguish the different algorithms

Hierarchical Clustering: Group Average

  • Compromise between Single and Complete Link
  • Strengths
  • Less susceptible to noise and outliers
  • Limitations
  • Biased towards globular clusters

Cluster Similarity: Ward’s Method

  • Similarity of two clusters is based on the increase in squared error when two clusters are merged
  • Similar to group average if distance between points is distance squared
  • Less susceptible to noise and outliers
  • Biased towards globular clusters
  • Hierarchical analogue of K-means
  • Can be used to initialize K-means

Hierarchical Clustering: Time and Space requirements

  • O(N2) space since it uses the proximity matrix.
  • N is the number of points.
  • O(N3) time in many cases
  • There are N steps and at each step the size, N2, proximity matrix must be updated and searched
  • Complexity can be reduced to O(N2 log(N) ) time for some approaches

Hierarchical Clustering: Problems and Limitations

  • Once a decision is made to combine two clusters, it cannot be undone
  • No objective function is directly minimized
  • Different schemes have problems with one or more of the following:
  • Sensitivity to noise and outliers
  • Difficulty handling different sized clusters and convex shapes
  • Breaking large clusters

DBSCAN

  • DBSCAN is a density-based algorithm.
  • Density = number of points within a specified radius (Eps)
  • A point is a core point if it has more than a specified number of points (MinPts) within Eps
  • These are points that are at the interior of a cluster
  • A border point has fewer than MinPts within Eps, but is in the neighborhood of a core point
  • A noise point is any point that is not a core point or a border point.

Cluster Validity

  • For supervised classification we have a variety of measures to evaluate how good our model is
  • Accuracy, precision, recall
  • For cluster analysis, the analogous question is how to evaluate the “goodness” of the resulting clusters?
  • But “clusters are in the eye of the beholder”!
  • Then why do we want to evaluate them?
  • To avoid finding patterns in noise
  • To compare clustering algorithms
  • To compare two sets of clusters
  • To compare two clusters

Determining the clustering tendency of a set of data, i.e., distinguishing whether non-random structure actually exists in the data.

Comparing the results of a cluster analysis to externally known results, e.g., to externally given class labels.

Evaluating how well the results of a cluster analysis fit the data without reference to external information.

– Use only the data

Comparing the results of two different sets of cluster analyses to determine which is better.

Determining the ‘correct’ number of clusters.

For 2, 3, and 4, we can further distinguish whether we want to evaluate the entire clustering or just individual clusters.

Different Aspects of Cluster Validation

  • Numerical measures that are applied to judge various aspects of cluster validity, are classified into the following three types.
  • External Index: Used to measure the extent to which cluster labels match externally supplied class labels.
  • Entropy
  • Internal Index: Used to measure the goodness of a clustering structure without respect to external information.
  • Sum of Squared Error (SSE)
  • Relative Index: Used to compare two different clusterings or clusters.
  • Often an external or internal index is used for this function, e.g., SSE or entropy
  • Sometimes these are referred to as criteria instead of indices
  • However, sometimes criterion is the general strategy and index is the numerical measure that implements the criterion.

Measures of Cluster Validity

  • Two matrices
  • Proximity Matrix
  • “Incidence” Matrix
  • One row and one column for each data point
  • An entry is 1 if the associated pair of points belong to the same cluster
  • An entry is 0 if the associated pair of points belongs to different clusters
  • Compute the correlation between the two matrices
  • Since the matrices are symmetric, only the correlation between
    n(n-1) / 2 entries needs to be calculated.
  • High correlation indicates that points that belong to the same cluster are close to each other.
  • Not a good measure for some density or contiguity based clusters.

Measuring Cluster Validity Via Correlation

  • Need a framework to interpret any measure.
  • For example, if our measure of evaluation has the value, 10, is that good, fair, or poor?
  • Statistics provide a framework for cluster validity
  • The more “atypical” a clustering result is, the more likely it represents valid structure in the data
  • Can compare the values of an index that result from random data or clusterings to those of a clustering result.
  • If the value of the index is unlikely, then the cluster results are valid
  • These approaches are more complicated and harder to understand.
  • For comparing the results of two different sets of cluster analyses, a framework is less necessary.
  • However, there is the question of whether the difference between two index values is significant

Framework for Cluster Validity

  • Cluster Cohesion: Measures how closely related are objects in a cluster
  • Example: SSE
  • Cluster Separation: Measure how distinct or well-separated a cluster is from other clusters
  • Example: Squared Error
  • Cohesion is measured by the within cluster sum of squares (SSE)
  • Separation is measured by the between cluster sum of squares

Where |Ci| is the size of cluster i

Internal Measures: Cohesion and Separation

“The validation of clustering structures is the most difficult and frustrating part of cluster analysis.

Without a strong effort in this direction, cluster analysis will remain a black art accessible only to those true believers who have experience and great courage.”

Algorithms for Clustering Data, Jain and Dubes

Final Comment on Cluster Validity

å

å

=

Î

=

K

i

C

x

i

i

x

m

dist

SSE

1

2

)

,

(

å

å

Î

=

i

C

x

i

i

m

x

WSS

2

)

(

å

=

i

i

i

m

m

C

BSS

2

)

(

Module5ProblemWorkbook.xlsx

P326-40

Capital budgeting
Note: All cash amounts are in $millions.
Cash outflows in various years, NPV of projects
Project 1 Project 2 Project 3 Project 4 Project 5 Project 6 Project 7 Project 8 Project 9 Project 10 Project 11 Project 12
Year 1
Year 2
Year 3
Year 4
Year 5
NPV
Selected projects (1 if selected, 0 if not)
Project 1 Project 2 Project 3 Project 4 Project 5 Project 6 Project 7 Project 8 Project 9 Project 10 Project 11 Project 12
Budget constraints
Outflow Budget
Year 1
Year 2
Year 3
Year 4
Year 5
Total NPV

Problem 5.10

P326-44

Choosing college courses
Requirements (along side) that are met by taking available courses (along top) (1 if it fills reqt, 0 if not)
Calculus OR Data Structures Bus Stats Simulation Intro to Comp Prog Forecasting
OR
Math
Computer
Calculus OR Data Structures Bus Stats Simulation Intro to Comp Prog Forecasting
Course taken
Prerequisite constraints
(Calculus for Bus Stats)
(Intro to Comp Prog for Simulation)
(Intro to Comp Prog for Data Structures)
(Bus Stats for Forecasting)
Major requirement constraints
Courses taken Required
OR
Math
Computer
Total courses

Solve the problems using excel-Optimization Models with Integer Variables

Review your understanding of the material by working through the problem sets. Pay particular attention to the problems indicated by the colored box around the problem number.  You can compare your work on those with the solutions that are in the Student Solutions manual.  You should find the Conclusion pages 324-325 a good review of what you should have learned in this module.

    40-You are given a group of possible investment projects for your companys capital. For each project, you
are given the NPV the project would add to the firm, as well as the cash outflow required by each project during each year. Given the information in the file P06_40.xlsx, determine the investments that maximize the firms NPV. The firm has $25 million available during each of the next five years. All numbers are in millions of dollars. 

44-To graduate from Southeastern University with a major in operations research (OR), a student must complete at least three math courses, at least three OR courses, and at least two computer courses. Some courses can be used to fulfill more than one requirement: Calculus can fulfill the math requirement; Operations Research can fulfill the math and OR requirements; Data Structures can fulfill the computer and math requirements; Business Statistics can fulfill the math and OR requirements; Computer Simulation can fulfill the OR and computer require- ments; Introduction to Computer Programming can fulfill the computer requirement; Forecasting can fulfill the OR and math requirements; and Data Mining can fulfill the OR, math, and computer requirements. Some courses are prerequisites for others: Calculus is a prereq- uisite for Business Statistics; Introduction to Computer Programming is a prerequisite for Computer Simulation and for Data Structures; and Business Statistics is a pre- requisite for Forecasting and Data Mining. Determine how to minimize the number of courses needed to satisfy the major requirements. (Hint: Because Calculus is a prerequisite for Business Statistics, for example, you will need a constraint that ensures that the decision variable for Calculus is greater than or equal to the decision vari- able for Business Statistics.) 

BackPainDoctorinRudrapur-ThemedicityRudrapur.pdf

�द मे�ड�सट�, ��पुर

Teen pani, Bigbara, Rudrapur, Uttarakhand 263153

www.themedicityrudrapur.com

Are you having

BACK PAIN

Regularstretching.

Correctyour posture.

Use Anti-inflammatory.

24 घंटे मे�डकल आपातकालीन क� सु�वधा उपल� है।

+91 7900771771CONTACT US:

SPC – Module 4

Think about the last time you needed to work in a small group. What was it you were supposed to do, a project, an activity, plan an event, solve a problem, etc.? Was the end result of your working together productive or unproductive? Why do you think this was so? Were you the leader of the group? If not, could you tell who the group leader was? How? Did the direction the group took in terms of completing the task align with how you thought the task should be completed? Why or why not?