`sourced.ml.core.extractors.literals`¶

Module Contents¶

class sourced.ml.core.extractors.literals.HashedTokenParser[source]¶

process_token(self, token)[source]¶

class sourced.ml.core.extractors.literals.Literals2Bag(token2index=None, token_parser=None)[source]¶

Bases:sourced.ml.core.algorithms.uast_ids_to_bag.UastIds2Bag

Converts a UAST to a bag-of-literals.

XPATH = //*[@roleLiteral][source]¶

class sourced.ml.core.extractors.literals.LiteralsBagExtractor(docfreq_threshold=None, **kwargs)[source]¶

Bases:sourced.ml.core.extractors.bags_extractor.BagsExtractor

NAME = lit[source]¶

NAMESPACE = l.[source]¶

OPTS[source]¶

uast_to_bag(self, uast)[source]¶