File: //lib/python3.6/site-packages/docutils/utils/__pycache__/code_analyzer.cpython-36.pyc
3
w2�e- � @ s� d Z ddlmZ y(ddlZddlmZ ddlmZ dZW n e e
fk
rT dZY nX dd d
gZ G d d � d e�Z G d
d� de
�ZG dd� de
�ZdS )z@Lexical analysis of formal languages (i.e. code) using Pygments.� )�ApplicationErrorN)�get_lexer_by_name)�_get_ttype_classTF�token�text� c @ s e Zd ZdS )�
LexerErrorN)�__name__�
__module__� __qualname__� r r �#/usr/lib/python3.6/code_analyzer.pyr s r c @ s* e Zd ZdZd
dd�Zdd� Zdd� Zd S ) �Lexera� Parse `code` lines and yield "classified" tokens.
Arguments
code -- string of source code to parse,
language -- formal language the code is written in,
tokennames -- either 'long', 'short', or '' (see below).
Merge subsequent tokens of the same token-type.
Iterating over an instance yields the tokens as ``(tokentype, value)``
tuples. The value of `tokennames` configures the naming of the tokentype:
'long': downcased full token type name,
'short': short name defined by pygments.token.STANDARD_TYPES
(= class argument used in pygments html output),
'none': skip lexical analysis.
�shortc C sr || _ || _|| _d| _|dks(|dkr,dS ts8td��yt| j�| _W n$ tjj k
rl td| ��Y nX dS )zE
Set up a lexical analyzer for `code` in `language`.
Nr r Znonez0Cannot analyze code. Pygments package not found.z6Cannot analyze code. No Pygments lexer found for "%s".)r r )
�code�language�
tokennames�lexer�
with_pygmentsr r �pygments�utilZ
ClassNotFound)�selfr r r r r r
�__init__/ s zLexer.__init__c c st t |�}t|�\}}x6|D ].\}}||kr4||7 }q||fV || }}qW |jd�rb|dd� }|rp||fV dS )zrMerge subsequent tokens of same token-type.
Also strip the final newline (added by pygments).
�
N� ���)�iter�next�endswith)r �tokensZlasttypeZlastval�ttype�valuer r r
�mergeG s
z Lexer.mergec c s� | j dkrg | jfV dS tj| j| j �}xV| j|�D ]H\}}| jdkr\t|�j� jd�}n
t |�g}dd� |D �}||fV q6W dS )z7Parse self.code and yield "classified" tokens.
NZlong�.c S s g | ]}|t kr|�qS r )�unstyled_tokens)�.0�clsr r r
�
<listcomp>e s z"Lexer.__iter__.<locals>.<listcomp>)
r r r Zlexr" r �str�lower�splitr )r r Z tokentyper! �classesr r r
�__iter__Y s
zLexer.__iter__N)r )r r
r �__doc__r r" r, r r r r
r s
r c @ s e Zd ZdZdd� Zdd� ZdS )� NumberLinesat Insert linenumber-tokens at the start of every code line.
Arguments
tokens -- iterable of ``(classes, value)`` tuples
startline -- first line number
endline -- last line number
Iterating over an instance yields the tokens with a
``(['ln'], '<the line number>')`` token added for every code line.
Multi-line tokens are splitted.c C s" || _ || _dtt|�� | _d S )Nz%%%dd )r � startline�lenr( �fmt_str)r r r/ Zendliner r r
r v s zNumberLines.__init__c c s� | j }dg| j| fV xh| jD ]^\}}|jd�}x<|d d� D ],}||d fV |d7 }dg| j| fV q@W ||d fV q W d S )N�lnr r r r )r/ r1 r r* )r �linenor r! �lines�liner r r
r, | s
zNumberLines.__iter__N)r r
r r- r r, r r r r
r. i s r. )r- Zdocutilsr r Zpygments.lexersr Zpygments.formatters.htmlr r � ImportError� SyntaxErrorr$ r �objectr r. r r r r
�<module> s
N