What is Lexical Analysis?? How its working?? Concept of tokens ??
16 December, 2015
- Lexical analyzer is divide the given sentence into meaningful string known as Tokens.
- Lexical Analyzer eliminates comment lines and whitespace character etc.
- Lexical analyzer helps to give error messages by providing line no. and column no.
- Lets see the structure of lexical analyzer:
- Here it will create / find tokens with the help of symbol table
We use regular expressions to describe tokens of a programming language.
Find no of tokens:-
int max (i,j)
{
int i, j;
return i;
}
Ans: No of tokens: int, max, (, i, , , j, ),{, int , i, , , j, ;, return, i, ;, } .
17
Que:- How many tokens a+++++b??
Ans:- 1. a
2. ++
3. ++
4. +
5. b
5 tokens but practically not possible
With these examples you can understand how to find out the tokens from program
