Showing posts with label Algorithms. Show all posts
Showing posts with label Algorithms. Show all posts

Sunday, September 20, 2015

Python Code for Maximum Sum Subarray Problem

"In computer science, the maximum subarray problem is the task of finding the contiguous subarray within a one-dimensional array of numbers (containing at least one positive number) which has the largest sum. For example, for the sequence of values −2, 1, −3, 4, −1, 2, 1, −5, 4; the contiguous subarray with the largest sum is 4, −1, 2, 1, with sum 6." --Wikipedia

A linear time solution was proposed by Jay Kadane and was hence forth named as Kadane's Algorithm. The algorithm paradigm followed is Dynamic Programming. Here I have given you the python code for this problem.



Continue Reading →

Thursday, July 2, 2015

Monday, June 29, 2015

Code for Damerau Lavenshtein Distance in Java

Earlier I had posted a tutorial on Fuzzy String Matching with Python Code. Here in this post I am sharing the Damerau Lavenshtein Algorithm Code in Java.

package com.codehackersblog.editdistance;

/**
*
* @author psychocoder
* @version 1.0
*/
public class DamerauLavenshtein {

    private String s1;
    private String s2;
    private int len_s1;
    private int len_s2;

    public DamerauLavenshtein() {
        s1 = null;
        s2 = null;
    }

    public DamerauLavenshtein(String s1, String s2) {
        this.s1 = s1.toLowerCase();
        this.s2 = s2.toLowerCase();
    }

    private void putValue(String m, String n) {
        s1 = m.toLowerCase();
        s2 = n.toLowerCase();
    }

    private int getMinimum(int a, int b, int c) {
        return Math.min(a, Math.min(b, c));
    }

    private int getDamerauLavenshtein() {

        len_s1 = s1.length();
        len_s2 = s2.length();

        int[][] d = new int[len_s1 + 1][len_s2 + 1];
        int i, j, cost;

        if (len_s1 == 0) {
            return len_s2;
        }

        if (len_s2 == 0) {
            return len_s1;
        }

        for (i = 0; i <= len_s1; i++) {
            d[i][0] = i;
        }

        for (j = 0; j <= len_s2; j++) {
            d[0][j] = j;
        }

        for (i = 1; i <= len_s1; i++) {
            for (j = 1; j <= len_s2; j++) {
                if (s1.charAt(i - 1) == s2.charAt(j - 1)) {
                    cost = 0;
                } else {
                    cost = 1;
                }

                d[i][j] = getMinimum(
                        d[i - 1][j] + 1, /*Deletion*/
                        d[i][j - 1] + 1, /*Insertion*/
                        d[i - 1][j - 1] + cost /*Substitution*/
                );

                if (i > 1 && j > 1 && s1.charAt(i - 1) == s2.charAt(j - 1)
                        && s2.charAt(j - 1) == s1.charAt(i - 1)) {
                    d[i][j] = Math.min(d[i][j], d[i - 2][j - 2] + cost);  /*transposition*/

                }
            }
        }

        return d[len_s1][len_s2];
    }

    private int getDamerauLavenshtein(String s1, String s2) {

        s1 = s1.toLowerCase();
        s2 = s2.toLowerCase();

        len_s1 = s1.length();
        len_s2 = s2.length();

        int[][] d = new int[len_s1 + 1][len_s2 + 1];
        int i, j, cost;

        for (i = 0; i <= len_s1; i++) {
            d[i][0] = i;
        }

        for (j = 0; j <= len_s2; j++) {
            d[0][j] = j;
        }

        for (i = 1; i <= len_s1; i++) {
            for (j = 1; j <= len_s2; j++) {
                if (s1.charAt(i - 1) == s2.charAt(j - 1)) {
                    cost = 0;
                } else {
                    cost = 1;
                }

                d[i][j] = getMinimum(
                        d[i - 1][j] + 1, /*Deletion*/
                        d[i][j - 1] + 1, /*Insertion*/
                        d[i - 1][j - 1] + cost /*Substitution*/
                );

                if (i > 1 && j > 1 && s1.charAt(i - 1) == s2.charAt(j - 1)
                        && s2.charAt(j - 1) == s1.charAt(i - 1)) {
                    d[i][j] = Math.min(d[i][j], d[i - 2][j - 2] + cost);  /*transposition*/

                }
            }
        }

        return d[len_s1][len_s2];

    }

    public static void main(String... args) {
        DamerauLavenshtein ob = new DamerauLavenshtein("Ludo", "lludo");
        System.out.println(ob.getDamerauLavenshtein());
        System.out.println(ob.getDamerauLavenshtein("Hello", "olhel"));
    }

}

Continue Reading →

Code for Wagner Fisher Algorithm in Java

Earlier I had posted a tutorial on Fuzzy String Matching with Python Code. Here in this post I am sharing the Wagner Fisher Algorithm Code in Java.

package com.codehackersblog.editdistance;

/**
*
* @author psychocoder
* @version 1.0
*/
public class WagnerFisher {

    private String s1;
    private String s2;
    private int len_s1;
    private int len_s2;

    public WagnerFisher() {
        s1 = null;
        s2 = null;
    }

    public WagnerFisher(String s1, String s2) {
        this.s1 = s1.toLowerCase();
        this.s2 = s2.toLowerCase();
    }

    private void putValue(String m, String n) {
        s1 = m.toLowerCase();
        s2 = n.toLowerCase();
    }

    private int getMinimum(int a, int b, int c) {
        return Math.min(a, Math.min(b, c));
    }

    private int getWagnerFisher() {

        len_s1 = s1.length();
        len_s2 = s2.length();

        int[][] d = new int[len_s1 + 1][len_s2 + 1];
        int i, j, cost;

        for (i = 0; i <= len_s1; i++) {
            d[i][0] = i;
        }

        for (j = 0; j <= len_s2; j++) {
            d[0][j] = j;
        }

        for (i = 1; i <= len_s1; i++) {
            for (j = 1; j <= len_s2; j++) {
                if (s1.charAt(i - 1) == s2.charAt(j - 1)) {
                    d[i][j] = d[i - 1][j - 1];
                } else {

                    d[i][j] = getMinimum(
                            d[i - 1][j] + 1, /*Deletion*/
                            d[i][j - 1] + 1, /*Insertion*/
                            d[i - 1][j - 1] + 1 /*Substitution*/
                    );
                }
            }
        }

        return d[len_s1][len_s2];
    }

    private int getWagnerFisher(String s1, String s2) {

        s1 = s1.toLowerCase();
        s2 = s2.toLowerCase();

        len_s1 = s1.length();
        len_s2 = s2.length();

        int[][] d = new int[len_s1 + 1][len_s2 + 1];
        int i, j, cost;

        for (i = 0; i <= len_s1; i++) {
            d[i][0] = i;
        }

        for (j = 0; j <= len_s2; j++) {
            d[0][j] = j;
        }

        for (i = 1; i <= len_s1; i++) {
            for (j = 1; j <= len_s2; j++) {
                if (s1.charAt(i - 1) == s2.charAt(j - 1)) {
                    d[i][j] = d[i - 1][j - 1];
                } else {

                    d[i][j] = getMinimum(
                            d[i - 1][j] + 1, /*Deletion*/
                            d[i][j - 1] + 1, /*Insertion*/
                            d[i - 1][j - 1] + 1 /*Substitution*/
                    );
                }
            }
        }

        return d[len_s1][len_s2];

    }

    public static void main(String... args) {
        WagnerFisher ob = new WagnerFisher("loookes", "looks");
        System.out.println(ob.getWagnerFisher());
        ob.putValue("Hiya", "Hey");
        System.out.println(ob.getWagnerFisher());
        System.out.println(ob.getWagnerFisher("Hello", "Heoll"));
    }

}

Continue Reading →

Sunday, June 28, 2015

C Code for Longest Common Subsequence Using Dynamic Programming

Here's the C Code for LCS using Dynamic programming :-


Output:-
Output:-

Matrix Generated

0 |0 |0 |0 |0 |
0 |0 |0 |0 |0 |
0 |1 |1 |1 |1 |
0 |1 |2 |2 |2 |
0 |1 |2 |3 |3 |
0 |1 |2 |3 |4 |

Length of Longest Common Subsequence = 4

Continue Reading →

Saturday, June 27, 2015

C Code for Knapsack Problem using Dynamic Programming

Knapsack problem
Knapsack problem (Photo credit: Wikipedia)
Knapsack Problem using Dynamic Programming.

Code:-


Sample Output:- 

Enter the number of elements : 3
Enter the profit and weights of the elements
Item no : 1
4 6
Item no : 2
1 2
Item no : 3
7 3
Enter the capacity
7
Items in the KnapSack are :
Sl.no    weight          profit
----------------------------------------
1        2                1
2        3                7
Total profit = 8
Continue Reading →

Friday, June 26, 2015

Introduction to Public Key Cryptography and RSA Encryption

Cryptographically secure pseudorandom number g...
Cryptographically secure pseudorandom number generator (Photo credit: Wikipedia)
Cryptography

The art of protecting information by transforming it (encrypting it) into an unreadable format, called cipher text. Only those who possess a secret key can decipher (or decrypt) the message into plain text. Encrypted messages can sometimes be broken by cryptanalysis, also called code breaking, although modern cryptography techniques are virtually unbreakable.

Cryptography systems can be broadly classified into:-

1. Symmetric-key Cryptography(or Secret Key Cryptography):-systems that use a single key that both the sender and recipient have.

2. Public-key Cryptography:-systems that use two keys, a public key known to everyone and a private key that only the recipient of messages uses.

3. Hash Functions: Uses a mathematical transformation to irreversibly "encrypt" information

See the Picture below :-

[Image: crypto_types.gif]

Today I will discuss with you Public Key Cryptography by taking RSA Encryption as an example:-

Public-Key Cryptography(PKC)

Public-key cryptography has been said to be the most significant new development in cryptography.Modern PKC was first described publicly by Stanford University professor Martin Hellman and graduate student Whitfield Diffie in 1976. Their paper described a two-key crypto system in which two parties could engage in a secure communication over a non-secure communications channel without having to share a secret key.

PKC depends upon the existence of so-called one-way functions, or mathematical functions that are easy to compute whereas their inverse function is relatively difficult to compute. Let me give you two simple examples:

Multiplication vs. factorization: Suppose I tell you that I have two prime numbers, 3 and 7, and that I want to calculate the product; it should take almost no time to calculate that value, which is 21. Now suppose, instead, that I tell you that I have a number, 21, and I need you tell me which pair of prime numbers I multiplied together to obtain that number. You will eventually come up with the solution but whereas calculating the product took milliseconds, factoring will take longer. The problem becomes much harder if I start with primes that have 400 digits or so, because the product will have ~800 digits.

Exponentiation vs. logarithms: Suppose I tell you that I want to take the number 3 to the 6th power; again, it is relatively easy to calculate 36 = 729. But if I tell you that I have the number 729 and want you to tell me the two integers that I used, x and y so that logx 729 = y, it will take you longer to find the two values.
While the examples above are trivial, they do represent two of the functional pairs that are used with PKC; namely, the ease of multiplication and exponentiation versus the relative difficulty of factoring and calculating logarithms, respectively. The mathematical "trick" in PKC is to find a trap door in the one-way function so that the inverse calculation becomes easy given knowledge of some item of information.

Generic PKC employs two keys that are mathematically related although knowledge of one key does not allow someone to easily determine the other key. One key is used to encrypt the plaintext and the other key is used to decrypt the ciphertext. The important point here is that it does not matter which key is applied first, but that both keys are required for the process to work (Figure 1B). Because a pair of keys are required, this approach is also called asymmetric cryptography.

In PKC, one of the keys is designated the public key and may be advertised as widely as the owner wants. The other key is designated the private key and is never revealed to another party. It is straight forward to send messages under this scheme. Suppose Alice wants to send Bob a message. Alice encrypts some information using Bob's public key; Bob decrypts the ciphertext using his private key. This method could be also used to prove who sent a message; Alice, for example, could encrypt some plaintext with her private key; when Bob decrypts using Alice's public key, he knows that Alice sent the message and Alice cannot deny having sent the message.

Examples of PKC :-
  • RSA
  • Diffie-Hellman
  • Digital Signature Algorithm(DSA)
  • Elliptic Curve Cryptography (ECC)
  • ElGamal

RSA

The first, and still most common, PKC implementation, named for the three MIT mathematicians who developed it — Ronald Rivest, Adi Shamir, and Leonard Adleman. RSA today is used in hundreds of software products and can be used for key exchange, digital signatures, or encryption of small blocks of data. RSA uses a variable size encryption block and a variable size key. The key-pair is derived from a very large number, n, that is the product of two prime numbers chosen according to special rules; these primes may be 100 or more digits in length each, yielding an n with roughly twice as many digits as the prime factors. The public key information includes n and a derivative of one of the factors of n; an attacker cannot determine the prime factors of n (and, therefore, the private key) from this information alone and that is what makes the RSA algorithm so secure.

Algorithm of RSA:-

To generate the encryption and decryption keys, we can

proceed as follows.

1. Generate randomly two “large” primes 'p' and 'q'.

2. Compute 'n' = pq and 'φ' = (p − 1)*(q − 1).

3. Choose a number 'e' so that gcd(e, φ) = 1.

4. Find the multiplicative inverse of 'e' modulo 'φ', i.e., find d so that
    e*d ≡ 1 (mod φ).

This can be done efficiently using Euclid’s Ex-tended Algorithm.

The encryption public key is KE = (n, e) and the decryption private key is KD = (n, d).

The encryption function is :- E(M ) = M^e mod n.
The decryption function is :- D(M ) = M^d mod n.
These functions satisfy D(E(M )) = M and E(D(M )) = M ,for any 0 ≤ M < n.
 

Now Let's take an example which will clear the facts :-

P = 61 <- first prime number (destroy this after computing E and D)
Q = 53 <- second prime number (destroy this after computing E and D)
PQ = 3233 <- modulus (give this to others)
E = 17 <- public exponent (give this to others)
D = 2753 <- private exponent (keep this secret!)


Your public key is (E,PQ).
Your private key is D.

The encryption function is:

encrypt(T) = (T^E) mod PQ = (T^17) mod 3233

The decryption function is:

decrypt(C) = (C^D) mod PQ= (C^2753) mod 3233

To encrypt the plaintext value 123, we do this:

encrypt(123) = (123^17) mod 3233 = 337587917446653715596592958817679803 mod 3233 = 855

To decrypt the ciphertext value 855, we do this:

decrypt(855) = (855^2753) mod 3233= 123

One way to compute the value of 855^2753 mod 3233 is like this:

2753 = 101011000001 base 2,
therefore, 2753 = 1 + 2^6 + 2^7 + 2^9 + 2^11= 1 + 64 + 128 + 512 + 2048

Consider this table of powers of 855:

855^1 = 855 (mod 3233)
855^2 = 367 (mod 3233)
855^4 = 367^2 (mod 3233) = 2136 (mod 3233)
855^8 = 2136^2 (mod 3233) = 733 (mod 3233)
855^16 = 733^2 (mod 3233) = 611 (mod 3233)
855^32 = 611^2 (mod 3233) = 1526 (mod 3233)
855^64 = 1526^2 (mod 3233) = 916 (mod 3233)
855^128 = 916^2 (mod 3233) = 1709 (mod 3233)
855^256 = 1709^2 (mod 3233) = 1282 (mod 3233)
855^512 = 1282^2 (mod 3233) = 1160 (mod 3233)
855^1024 = 1160^2 (mod 3233) = 672 (mod 3233)
855^2048 = 672^2 (mod 3233) = 2197 (mod 3233)


Given the above, we can do like this:

855^2753 (mod 3233)
= 855^(1 + 64 + 128 + 512 + 2048) (mod 3233)
= 855^1 * 855^64 * 855^128 * 855^512 * 855^2048 (mod 3233)
= 855 * 916 * 1709 * 1160 * 2197 (mod 3233)
= 794 * 1709 * 1160 * 2197 (mod 3233)
= 2319 * 1160 * 2197 (mod 3233)
= 184 * 2197 (mod 3233)
= 123 (mod 3233)
= 123



References :

http://www.garykessler.net/library/crypto.html
http://plansoft.org/wp-content/uploads/knowledge/inne/RSA.pdf


Thank You. Hope you like this tutorial. Share it among your friends :)


Continue Reading →

Sunday, June 21, 2015

Fuzzy String Matching or Searching with Python Code

[Image: hoRwHam.png]
Hello everyone, I have been recently reading about Text Mining and Natural Language Processing (NP). So, I will be sharing a few articles on TextMining from now on. So, that you can have an interest in this beautiful field of Computer Science.

Prerequisites

In order to understand the main content of the article you need some prerequisites which are as follows :- 

  • Familiar with Programming.
  • Know at least one programming language.
  • Basics of Python (Optional but Recommended)
  • Have some prior exposure to Data Structures and algorithmic techniques like Backtracking, Dynamic Programming etc.
  • Have some gray cells and have patience.
  • Passion for Computer Science.
Let's Start,


The first question that should arise in your mind when you hear the term TextMining is What is TextMining and What does it do ? hence we will understand first what is Text or Data Mining. Lets proceed.

Text Mining or Data Mining

Text mining, also referred to as text data mining, roughly equivalent to text analytics, refers to the process of deriving high-quality information from text. High-quality information is typically derived through the devising of patterns and trends through means such as statistical pattern learning.

Text mining usually involves the process of structuring the input text (usually parsing, along with the addition of some derived linguistic features and the removal of others, and subsequent insertion into a database), deriving patterns within the structured data, and finally evaluation and interpretation of the output.

To explain it simply

Data Mining is a processing of locating or finding important expression, phrases or sentences or events from data or text which can be further processed to get more useful information. Even though data mining tries to collect information from data, its different from Information Retrieval, instead of the fact that Data Mining, Information Retrieval or Natural Language Processing are often connected together and misunderstood as one, I will explain how they are different some other time and not here. If you wanna develop an application based on Text or where you have to deal with large amount of data and you need an intelligent system which will classify the data and sorts them into different categories or do different processes then you need these concepts a lot.

Real life examples of Data Mining

1. Google Search : Whenever you starting writing a query it automatically suggests the possible search queries. How does it do ? They use these concept to derive the most probable search query.

2. Spell Correction : You might have seen that if you type wrong spelling's most search engines or word processors mark them with a red underline which means that you have a spelling error. MS Word or Android apps has automatic spell correction. They use these NLP and Text Mining concepts. See the image it will clear a lot of doubts. See how Google does spell correction and the search results are an example of the query I entered.



3. Amazon or Online Shopping Stores : Most of you must have done online shopping a lot. Whenever you visit a site for the first time and search for some specific product. You see some thing written as recommended for you. Next time when you visit that site you might be astonished to see that the products you see when you open the webpage are similar to what you had searched in your last visit but more refined. So, how does it work ? the earlier mentioned concepts are used for these. They collect your browsing history and process the data and finally show you the most recommended products.

There are thousands of examples just Google for it :P
I hope by now you have somewhat understood what is data or text mining. Now we will come to today's real discussion that is fuzzy string searching. So lets continue.



Today's Discussion

In this section we discuss about String matching or string similarity algorithms. We will discuss the following algorithms today.

1. Damerau-Lavenshtein Distance.
2. Levenshtein Distance.
3. Wagner-Fisher Algorithm.


Strings are among the most important elements of computer science. There are literally 100's of research papers on strings. String searching and matching is one of he most important part in Text Processing and text analysis, since all the data to intelligent systems are fed as string. Strings are nothing but a sequence of characters joined together.

Fuzzy String Searching or Fuzzy String Matching

Fuzzy string search algorithms are algorithms that are used to match either exactly or partially of one string with another string. So, what exactly does fuzzy mean ? 

Fuzzy by the word we can understand that elements that aren't clear or is like an illusion. Therefore, in short we say that when there is a set of n elements and another set of m elements and they both have partially same elements then we can say that the relation between them is fuzzy.

(Funny example Most women have a fuzzy thinking that their men are cheating on them but instead of taking it as fuzzy they take it for granted :P)

By Fuzzy string searching (also known as approximate string matching) we identify those strings which are similar to a set of other strings. A number of conclusions can be drawn from these like Similarity co-efficient, similar words etc, etc.

The problem of approximate string matching is typically divided into two sub-problems: finding approximate sub-string matches inside a given string and finding dictionary strings that match the pattern approximately.

Fuzzy search algorithms (also known as similarity search algorithms) serve as a base for spell-checkers and search engines like Google or Bing or Yahoo etc. For example, these algorithms are used to provide the "Did you mean ..." functionality that is very common.

Example :- 

If you ask a machine to find the similarity between two words and correct them if required, say, "Looks" and "Lookes", the machine will return true for the first string "Looks" as its correct and will return the corrected string as "Looks" from the erroneous string "Lookes".

Simple Formulation of Fuzzy String Problems :-

"Find in the text or dictionary of a finite size n for all the words that match a word given as input (or starts with the given word), taking into account m possible differences (errors)."

Fuzzy search algorithms are characterized by metric - a function of distance between two words, which provides a measure of their similarity. A strict mathematical definition of metric includes a requirement to meet triangle inequality (p - metric function, Z - a set of words):

[Image: dbh35lX.png]

LaTeX code for above equation.



p(x,y)  \leq p(x,k) + p(k,y), where,  (x,y,k)  \varepsilon  Z

What is Edit-Distance ?

Definition From Wikipedia

Edit distance is a way of quantifying how dissimilar two strings (e.g., words) are to one another by counting the minimum number of operations required to transform one string into the other. Edit distances find applications in natural language processing, where automatic spelling correction can determine candidate corrections for a misspelled word by selecting words from a dictionary that have a low distance to the word in question.
Simply we can say that - The closeness of a match is measured in terms of the number of primitive operations necessary to convert the string into an exact match. This number is called the edit distance between the string and the pattern.

The primitive operations performed are, Insertion, Deletion, Substitution
. Example :- 

1. Mad -- > Made, Insertion Operation Performed
2. Loooks -- > Looks, Deletion Operation Performed
3. Lick -- > Sick, Substitution Operation Performed

Now we will move to discuss the algorithms.



Damerau-Levenshtein Distance

Damerau-Levenshtein Distance is a distance (string metric) between two Strings, say String A and String B, which gives the minimum number of edit operations need to perform to transform String A to String B. Damerau's Algorithm can be used for spell correction with atmost 1 edit-distance.

There are four edit operations that can be performed with this algorithm. They are - 1. Insertion, 2. Deletion, 3. Substitution, and 4. Transposition. We have already seen the operation of the first three algorithms and so let me explain the last operation, that is transposition.

Transposition is the swapping of adjacent characters to bring them to a certain form.
Example :-

String A := olko
String B := look

When the above two strings are feed to a machine that computes the Damerau-Lavenshtein Distance find that the edit distance is two. 

So how it operates ?
We have two string transpositions here :-

1.) olko -> loko 
2.) loko -> look

Hence we get the final result as :- look
And the number of edits performed is : 2

Please Note that Damerau-Levenshtein Distance doesn't follow the Triangle inequality. You can see that in the image below :-

[Image: MnpRetD.jpg]

"All Theory and No Code Makes Psycho a Dull guy"

I am giving the working function for now. You can see the Psuedo code on Wiki, its is well written.

Here's the code in Python. I would encourage you to write it by yourself too.
def damerau_levenshtein(s1, len_s1, s2, len_s2):
    """
    A function to calculate the Damerau-Levenshtein Distance.

    :param s1: Parent String
    :param len_s1: Length Of s1
    :param s2: pattern String
    :param len_s2: length of s2
    :rtype : int returns the damerau-levenshtein distance
    """

    d = {}
    len_s1 = len(s1)
    len_s2 = len(s2)
    for i in range(-1, len_s1 + 1):
        d[i, -1] = i + 1
    for j in range(-1, len_s2 + 1):
        d[-1, j] = j + 1

    for i in range(len_s1):
        for j in range(len_s2):
            if s1[i] == s2[j]:
                cost = 0
            else:
                cost = 1
                
            d[(i, j)] = min(
                d[i - 1, j] + 1,  # Deletion
                d[i, j - 1] + 1,  # Insertion
                d[i - 1, j - 1] + cost,  # Substitution
            )

            if i and j and s1[i] == s2[j - 1] and s1[i - 1] == s2[j]:
                d[i, j] = min(d[i, j], d[i - 2, j - 2] + cost)  # transposition
    return d[len_s1 - 1, len_s2 - 1]


You can get the complete source code here : https://github.com/AnimeshShaw/TextMin...nshtein.py
To see the Code in Execution :- http://code.hackerearth.com/0356acE
You can visualize how the steps or the algorithm works here : Click Me

Levenshtein Distance

Levenshtein distance is also a string metric calculate the edit-distance between two strings. They have found a lot of applications and in most of the real world systems an efficient and modified form of Levenshtein distance is used including various NLP toolkits like LingPipe or NLTK. or GATE. They use different data structures which reduces the time complexity.

The only difference between Lavenshtein and Damerau-Lavenshtein is that this algorithm doesn't supports transposition and we have only can perform only three operations namely, insertion, deletion and substitution.

Mathematically, it can be written as :-

[Image: W82K5YX.jpg]

LaTeX code for above equation

\qquad\operatorname{lev}_{s,t}(i,j) = \begin{cases}
  \max(i,j) \text{ if} \min(i,j)=0, \\
  \min \begin{cases}
          \operatorname{lev}_{s,t}(i-1,j) + 1 \\
          \operatorname{lev}_{s,t}(i,j-1) + 1 \\
          \operatorname{lev}_{s,t}(i-1,j-1) + 1_{(s_i \neq t_j)}
       \end{cases} \text{ else,}
\end{cases}

Another significant difference between Levenshtein Distance and Damerau-Levenshtein Distance is that the former satisfies the Triangle Inequality whereas the latter does not. It is visible from the image given below where the numbers represent the edit distance between two nodes :-
[Image: 9aRw1lA.jpg]

Okay, lets do some programming now. We can follow the above math relation an we can formulate a recursive solution :-

def recursivelevenshtein(s1, len_s1, s2, len_s2):
    """
    A function to calculate the Levenshtein Distance using recursion.

    :param s1: Parent String
    :param len_s1: Length Of s1
    :param s2: pattern String
    :param len_s2: length of s2
    :rtype : int returns the levenshtein distance
    """
    if len_s1 == 0:
        return len_s2
    if len_s2 == 0:
        return len_s1
    return min(
        recursivelevenshtein(s1, len_s1 - 1, s2, len_s2) + 1,  # Deletion
        recursivelevenshtein(s1, len_s1, s2, len_s2 - 1) + 1,  # Insertion
        recursivelevenshtein(s1, len_s1 - 1, s2, len_s2 - 1) + (s1[len_s1 - 1] != s2[len_s2 - 1])  # Substitution
    )

Full Source :- https://github.com/AnimeshShaw/TextMin...nshtein.py
See Execution :- http://code.hackerearth.com/4ac2f4u

(I haven't given the visualize python link here, but you can do it by yourself)

I hope you understood the process. But this backtracking technique is very very inefficient for larger strings. As the complexity is cubic. The space complexity is large. In order to improve the performance we use the Wagner-Fisher algorithm.

Wagner-Fisher Algorithm
Wagner-Fisher algorithm presents before us a Dynamic Programming approach to solve the Levenshtein Distance Problem in Quadratic time, O(m * n) where m and n are the length two input strings respectively.

Lets get to the code already :-

def wagner_fisher(s1, len_s1, s2, len_s2):
    """
    A function to calculate the Levenshtein Distance using
    Wagner-Fisher Algorithm.

    :param s1: Parent String
    :param len_s1: Length Of s1
    :param s2: pattern String
    :param len_s2: length of s2
    :rtype : int returns the levenshtein distance
    """
    d = {}

    #Make to use O(min(n,m)) space
    if len_s1 > len_s2:
        s1, s2 = s2, s1
        len_s1, len_s2 = len_s2, len_s1
    for i in range(-1, len_s1 + 1):
        d[(i, -1)] = i + 1
    for j in range(-1, len_s2 + 1):
        d[(-1, j)] = j + 1

    for i in range(len_s1):
        for j in range(len_s2):
            if s1[i] == s2[j]:
                d[i, j] = d[i - 1, j - 1]
            else:
                d[(i, j)] = min(
                    d[(i - 1, j)] + 1,  # deletion
                    d[(i, j - 1)] + 1,  # insertion
                    d[(i - 1, j - 1)] + 1,  # substitution
                )
    return d[len_s1 - 1, len_s2 - 1]

Full Source :- https://github.com/AnimeshShaw/TextMin...-Fisher.py
See Execution :- http://code.hackerearth.com/75dc9aJ

(I haven't given the visualize python link here, but you can do it by yourself)

Lets See how the lookup matrix is generated for the strings : "LAD" and "LLUDO"
Here's the final Matrix, just follow the algorithm step[ by step and you should get the following matrix, if you don't then you have made some mistakes.



+------+---+---+---+---+
| S1 → |   | L | A | D |
+------+---+---+---+---+
| S2 ↓ | 0 | 1 | 2 | 3 |
+------+---+---+---+---+
| L    | 1 | 0 | 1 | 2 |
+------+---+---+---+---+
| L    | 2 | 0 | 1 | 2 |
+------+---+---+---+---+
| U    | 3 | 1 | 1 | 2 |
+------+---+---+---+---+
| D    | 4 | 2 | 2 | 2 |
+------+---+---+---+---+
| O    | 5 | 3 | 3 | 3 |
+------+---+---+---+---+
The bottom-right element gives the Levenshtein edit distance. This process is better than recursive. Now my question to you is that, Can we improve further. Well I leave this to you. Find a method and write a code to find the edit distance using this algorithm which performs better than the present algorithm. Comment in the section below.

Using NLTK

If you're using NLTK the you can calculate it using :-

Conclusion

So, all folks we have come to an end of our first encounter with text Processing. If you made it till the end you probably have understood a nice algorithm. I will be posting more articles on these topics as soon as I have time. If you have some doubt then please ask but before that go through the tutorial properly and execute the codes by yourself and practice them by yourself.

References

http://en.wikipedia.org/wiki/Levenshtein_distance
http://en.wikipedia.org/wiki/Wagner-Fischer_algorithm
http://en.wikipedia.org/wiki/Fuzzy_string_searching
http://en.wikipedia.org/wiki/Damerau%E2%...n_distance
https://www8.cs.umu.se/kurser/TDBA77/VT0...NODE46.HTM

Online Tools Used

Online LaTeX : http://www.sciweavers.org/free-online-latex-equation-editor
Draw Diagrams : https://www.draw.io/
Text Table Generator : http://www.tablesgenerator.com/text_tables

Credits to @XxGreenLanternxX to header banner image.


Thank you,
Sincerely,
Psycho_Coder.

Continue Reading →

Friday, June 19, 2015

Learning About the Stack Data Structure

Simple representation of a stack
Simple representation of a stack (Photo credit: Wikipedia)
Hello Everyone,

Today we will study about the data structure called Stack. But before roceeding with the tutorial I would like to read the following readers note.

Reader's Note :
Now we are good to start, So, Let's Begin.


Stack : A stack is a particular kind of abstract data type or collection in which the principal (or only) operations on the collection are the addition of an entity to the collection. The operations that are performed on the the stack are push which means to insert an element at the top of the stack and pop which means to remove the top element from the stack.
Stack follows LIFO (Last In First Out) Principle, that is the element which is inserted into a collection last is taken out first from the collection as well.
The concept of stack data structure is from from development point of view and it has found numerous applications, include system level design and making efficient processes.

Applications of Stack
 
1. Data Parsing
2. Compiler Designing.
3. Recursion
4. Solving numerous Classical problems like Tower of Hanoi.
5. System Memory Management.
6. Reversing String.
7. Mathematical Expression evaluation. (Converting them from infix to postfix and then manipulating evaluating them)
8. Backtracking.
etc...
 
Stack as an Abstract datatype


StackADT {

push(item) : insert the item into the stack


pop() : removes the top element.
size() : Returns the size of the stack.
isEmpty() : checks whether the stack is empty or not.
isFull() : checks whether the stack is full or not.
}
Now let us understand the concept of stack better by taking an example :-
 
Suppose we consider an array of the following elements :-

5 <--- Top element
6
7
8
9
 
Now we push another number onto the stack, say 34, then our stack has the following structure :-

34 <--- Top element.
5
6
7
8
9
 
So it is evident that the most recently inserted items are push onto the top of the stack following the LIFO principle. Again we push another number into the stack, say 2, just following the above concept we have the following :-

2 <--- Top element.
34
5
6
7
8
9
 
We have understood the push operation and so lets see how does pop works. Lets pop once from the above array and the array becomes :-

34 <--- Top element.
5
6
7
8
9
 
As you can see the top element was poped from the stack and the next element that is 34 becomes the top element. Okay so lets pop two more elements and then the stack has the following structure :-

6 <--- Top element.
7
8
9
 
What happened above ?
 
First 34 is poped and then 5 becomes the top element and then another element i.e. 5 is poped and 6 becomes the top element.
I hope you have understood the concept of stack. Since we have understood the concept of stack and so lets write some code now. I will use Java to write the code. We create an interface to understand the concept of our ADT.

StackInterface.java


JStackArray.java

A Visual or Animated Representation of Stack can be found here

Okay, We are now ready to study some applications of Stack. So lets begin.

Example of an Application of Stack

String Reverse.

A Stack can be used to reverse a list, integer, string etc. Lets Look at the following example to understand the concept better :-

Suppose we have a string variable NAME and a string value of "Firun" assigned to it.

String NAME = "Firun";

Now since Stack follows the LIFO principle so if we insert every character from NAME into the stack then we will have 'n' as the top element. Then we can pop the elements we will have our Strings reversed. Look at the following working :-

1. Insert 'F' into Stack. We have

'F' <--- Top

2. Insert 'i' into Stack. We have

'i' <--- Top
'F'

3. Subsequently we insert the remaining characters as well and we get the following Stack structure.

'n' <--- Top
'u'
'r'
'i'
'F'

Now we pop all the elements from the stack and we have the following sequence :-

'u' <--- Top
'r'
'i'
'F'

We get 'n'

'r' <--- Top
'i'
'F'

We get 'u'. Also initially we had 'n' and so now we add 'u' to it and we get "nu". Proceeding in a similar way we get the final string as "nuriF".

I hope you understood the logic. So lets peep at the function code :-

private String reverseString(){
       String m="";
       for(int i=0;i<str.length();i++){
           stack.push(str.charAt(i));
       }
       for(int i=0;i<str.length();i++){
          m=m+stack.pop();
       }
       return m;
   }

Algorithm

PROCEDURE REVERSE_STRING(M) {
    INITIALISE LEN = LENGTH(M)
    FOR I TO LEN:
        STACK.PUSH(M[I])
    FOR I TO LEN:        
        N = N + STACK.POP()
    RETURN N
}

If you are able to make it to this end then it probably means that you have read the tutorial. I hope you enjoyed it. If you have any questions or doubts then feel free to ask.

Stacks are very useful and they reduce your workload, but it solely depends on the programmer how efficiently he uses these concepts. More applications of Stack like Conversion to different notations and Tower of Hanoi will be covered separately.

Feedback would be appreciated. Comment in the comment section below.


Thank you,
Sincerely,
Psycho_Coder.
Continue Reading →

Follow Me!

Followers

Visitor Map