String Tokenizer in C++ using strtok()

By Divyanshu Pandey

In this Packet, we will learn about strtok() function in C++. The function is present in the cstring header file.

Strtok()

The function is used to split the string into tokens(smaller strings) according to delimiters. The function can be called in a loop to obtain tokens of the string.

The syntax of this function is as follows:-

char* strtok( char* str, const char* delimiter );

Parameters of strtok()

str - a string that is tokenized by parsing into tokens.

delimiter - a string that contains the delimiter character, which could vary from a particular call to another.

Return value of strtok()

On the first call of the strtok() function, it returns a pointer to the first token in str. In subsequent calls to the string, the strtok() function returns a pointer to the next token in the string, and every token is terminated with a null. Finally, a null pointer is returned when there are no tokens left. 

Now Let’s have a look at the following program to understand the working of the strtok() function :

For eg:-

#include 
#include 
using namespace std;

int main()
{
    char str[] = "This,is,a,packet";

    char delimiters[] = ",";

    cout << "Splitting the string into tokens:" <<endl;

    char *token_ptr;

    token_ptr=strtok(str,delimiters);

    while (token_ptr != NULL)
    {
        cout<<endl;
        cout <<"splitted string --> "<

The output of the above program:-