None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.

class BartKeywordGenerator[source]

BartKeywordGenerator(model_name, use_cuda=False)

Bart based keyword generator using huggingface transformers

BartKeywordGenerator is a base class for keyword generator. It is implemented based on huggingface transformer lib.

It has two function:

1. generate(): Given text input it will generate keywords. The parameters are based on transformers .generate arguments.
2. batch_generate(): Given a list of text inputs. Firstly it will split into batches and then generate.

class ExtractiveKeywordGenerator[source]

ExtractiveKeywordGenerator(use_cuda=False) :: BartKeywordGenerator

It will generate extractive keywords using bart based fined tunned model on openkp datasets

ExtractiveKeywordGenerator implements BartKeywordGenerator for extractive keyword generator

class AbstractiveKeywordGenerator[source]

AbstractiveKeywordGenerator(use_cuda=False) :: BartKeywordGenerator

It will generate abstractive keywords using bart based fined tunned model on kpTimes dataset

AbstractiveKeywordGenerator implements BartKeywordGenerator for abstractive keyword generator

extractive_generator = ExtractiveKeywordGenerator()

Some weights of the PyTorch model were not used when initializing the TF 2.0 model TFBartForConditionalGeneration: ['lm_head.weight']
- This IS expected if you are initializing TFBartForConditionalGeneration from a PyTorch model trained on another task or with another architecture (e.g. initializing a TFBertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing TFBartForConditionalGeneration from a PyTorch model that you expect to be exactly identical (e.g. initializing a TFBertForSequenceClassification model from a BertForSequenceClassification model).
All the weights of TFBartForConditionalGeneration were initialized from the PyTorch model.
If your task is similar to the task the model of the checkpoint was trained on, you can already use TFBartForConditionalGeneration for predictions without further training.

abstractive_generator = AbstractiveKeywordGenerator()

Some weights of the PyTorch model were not used when initializing the TF 2.0 model TFBartForConditionalGeneration: ['lm_head.weight']
- This IS expected if you are initializing TFBartForConditionalGeneration from a PyTorch model trained on another task or with another architecture (e.g. initializing a TFBertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing TFBartForConditionalGeneration from a PyTorch model that you expect to be exactly identical (e.g. initializing a TFBertForSequenceClassification model from a BertForSequenceClassification model).
All the weights of TFBartForConditionalGeneration were initialized from the PyTorch model.
If your task is similar to the task the model of the checkpoint was trained on, you can already use TFBartForConditionalGeneration for predictions without further training.

extractive_generator.generate(input_text)

[{'keywords': ['The death toll in Germany', ' Belgium', ' historic flood']}]

extractive_generator.batch_generate([input_text, input_text])

[{'keywords': ['The death toll in Germany', ' Belgium', ' historic flood']},
 {'keywords': ['The death toll in Germany', ' Belgium', ' historic flood']}]

abstractive_generator.generate(input_text)

[{'keywords': ['Floods', 'Germany', 'Belgium', 'Europe']}]

extractive_generator.generate(input_text, min_length=10, num_beams=5, 
    early_stopping=True, max_length=50)

[{'keywords': ['The death toll in Germany', ' Belgium', ' historic flood']}]

extractive_generator.generate(input_text, do_sample=True, max_length=50, top_k=0)

[{'keywords': ['The death toll in Germany', ' Belgium', ' historic flood']}]