Coders Packet

Tokenizing String in Java using Built-in Function

By Priyonath Mondal

The String Tokenizer class allows an application to break a string into tokens. A token is returned by taking a substring of the string that was used to create the StringTokenizer object.

String Tokenizer is a class that breaks a string into multiple tokens. Let us consider a string as, “JAVA PYTHON SQL AI”.

Now from the above string assume we have to parse wherever there are spaces to create smaller strings. That is, these are the smaller strings that has to be created:

JAVA

PYTHON

SQL

AI


All these smaller strings which are created after parsing a string are only called as “Tokens”.

We will make use of a built-in class called "StringTokenizer".

Things to be known about StringTokenizer:
Delimiters are those characters which separates the tokens.

Object of StringTokenizer can be created in three different ways:
• StringTokenizer st = new StringTokenizer( String str );
str is the string to be tokenized. This case, considers the default delimiter as space.
• StringTokenizer st = new StringTokenizer( String str, String delim );
delimeters is set of delimiters whcih are used to tokenize a string.
• StringTokenizer st = new StringTokenizer(String str, String delim, boolean flag);

Download project

Reviews Report

Submitted by Priyonath Mondal (priyonath)

Download packets of source code on Coders Packet