evalquiz_proto.shared package

Subpackages

• evalquiz_proto.shared.generated package
  • Module contents
    • Batch
      • Batch.capabilites
      • Batch.lecture_materials
      • Batch.question_to_generate
    • BatchStatus
      • BatchStatus.error_message
      • BatchStatus.module_status
      • BatchStatus.pipeline_module
    • ByMetrics
      • ByMetrics.evaluation_reference
      • ByMetrics.evaluation_result
      • ByMetrics.evaluator_type
    • Capability
      • Capability.educational_objective
      • Capability.keywords
      • Capability.relationship
    • Categorical
      • Categorical.categories
    • Complete
    • CourseSettings
      • CourseSettings.advantageous_capabilities
      • CourseSettings.course_goals
      • CourseSettings.required_capabilites
    • EducationalObjective
      • EducationalObjective.ANALYZE
      • EducationalObjective.APPLY
      • EducationalObjective.EVALUATE
      • EducationalObjective.INNOVATE
      • EducationalObjective.KNOW_AND_UNDERSTAND
      • EducationalObjective.SYNTHESIZE
    • Empty
    • Evaluation
      • Evaluation.language_model_evaluation
    • EvaluationResult
      • EvaluationResult.float_value
      • EvaluationResult.str_value
    • EvaluationResultType
      • EvaluationResultType.categorical
      • EvaluationResultType.value_range
    • EvaluationSettings
      • EvaluationSettings.metrics
    • GenerationEvaluationResult
      • GenerationEvaluationResult.evaluation_result
      • GenerationEvaluationResult.generation_result
    • GenerationResult
      • GenerationResult.multiple_choice
      • GenerationResult.multiple_response
    • GenerationSettings
      • GenerationSettings.mode
      • GenerationSettings.model
    • InternalConfig
      • InternalConfig.batches
      • InternalConfig.course_settings
      • InternalConfig.evaluation_settings
      • InternalConfig.generation_settings
      • InternalConfig.material_server_urls
    • LanguageModelEvaluation
      • LanguageModelEvaluation.evaluation_description
      • LanguageModelEvaluation.evaluation_result_type
      • LanguageModelEvaluation.few_shot_examples
      • LanguageModelEvaluation.model
    • LectureMaterial
      • LectureMaterial.file_type
      • LectureMaterial.hash
      • LectureMaterial.page_filter
      • LectureMaterial.reference
      • LectureMaterial.url
    • ListOfStrings
      • ListOfStrings.values
    • MaterialServerBase
      • MaterialServerBase.delete_material()
      • MaterialServerBase.get_material()
      • MaterialServerBase.get_material_hashes()
      • MaterialServerBase.get_material_name()
      • MaterialServerBase.upload_material()
    • MaterialServerStub
      • MaterialServerStub.delete_material()
      • MaterialServerStub.get_material()
      • MaterialServerStub.get_material_hashes()
      • MaterialServerStub.get_material_name()
      • MaterialServerStub.upload_material()
    • MaterialUploadData
      • MaterialUploadData.data
      • MaterialUploadData.metadata
    • Metadata
      • Metadata.mimetype
      • Metadata.name
    • Metric
      • Metric.evaluation
      • Metric.mode
      • Metric.reference
    • Mode
      • Mode.by_metrics
      • Mode.complete
      • Mode.overwrite
    • ModuleStatus
      • ModuleStatus.FAILED
      • ModuleStatus.IDLE
      • ModuleStatus.RUNNING
      • ModuleStatus.SUCCESS
    • MultipleChoice
      • MultipleChoice.answer_text
      • MultipleChoice.distractor_text
      • MultipleChoice.question_text
    • MultipleResponse
      • MultipleResponse.answer_texts
      • MultipleResponse.distractor_texts
      • MultipleResponse.question_text
    • Overwrite
    • PageFilter
      • PageFilter.lower_bound
      • PageFilter.upper_bound
    • PipelineModule
      • PipelineModule.input_datatype
      • PipelineModule.name
      • PipelineModule.output_datatype
    • PipelineResult
      • PipelineResult.internal_config
    • PipelineServerBase
      • PipelineServerBase.iterate_config()
    • PipelineServerStub
      • PipelineServerStub.iterate_config()
    • PipelineStatus
      • PipelineStatus.batch_status
      • PipelineStatus.result
    • Question
      • Question.evaluation_results
      • Question.generation_result
      • Question.question_type
    • QuestionType
      • QuestionType.MULTIPLE_CHOICE
      • QuestionType.MULTIPLE_RESPONSE
    • Relationship
      • Relationship.COMPLEX
      • Relationship.DIFFERENCES
      • Relationship.ORDER
      • Relationship.SIMILARITY
    • String
      • String.value
    • ValueRange
      • ValueRange.lower_bound
      • ValueRange.upper_bound

Submodules

evalquiz_proto.shared.exceptions module

exception evalquiz_proto.shared.exceptions.DataChunkNotBytesException

Bases: Exception

All chunks of a lecture material upload stream except the first one should be bytes.

exception evalquiz_proto.shared.exceptions.EvaluationNotCompatibleWithInternalEvaluation

Bases: Exception

Evaluation is not compatible with InternalEvaluation. Please check the QuestionEvaluation.internal_evaluations mappings.

exception evalquiz_proto.shared.exceptions.FileOverwriteNotPermittedException

Bases: Exception

A file is already present at the given location and is not permitted to be overwritten.

exception evalquiz_proto.shared.exceptions.FirstDataChunkNotMetadataException

Bases: Exception

The first chunk of a lecture material upload stream should be of type Metadata.

exception evalquiz_proto.shared.exceptions.LectureMaterialLocallyNotFoundException

Bases: Exception

The requested lecture material cannot be provided by the InternalMaterialController.

exception evalquiz_proto.shared.exceptions.LectureMaterialNotFoundOnRemotesException

Bases: PipelineExecutionException

All remotes have been requested to provide the lecture material, but none of the remotes is able to.

exception evalquiz_proto.shared.exceptions.MimetypeMismatchException

Bases: Exception

Expected mimetype does not match the given mimetype.

exception evalquiz_proto.shared.exceptions.MimetypeNotDetectedException

Bases: Exception

Mimetype could not have been detected.

exception evalquiz_proto.shared.exceptions.MissingDefaultInternalConfigAttributeException

Bases: Exception

A DefaultInternalConfig attribute was not be found.

exception evalquiz_proto.shared.exceptions.NoMimetypeMappingException

Bases: Exception

The system could not map any file extension to the given mimetype, the mimetype could be invalid.

exception evalquiz_proto.shared.exceptions.PipelineExecutionException

Bases: Exception

Unexpected behavior occurred in the execution of a PipelineModule

exception evalquiz_proto.shared.exceptions.PipelineModuleCompositionNotValidException

Bases: Exception

Input and output datatypes at least one successive PipelineModule are not compatible.

exception evalquiz_proto.shared.exceptions.PipelineModuleRuntimeInputException

Bases: Exception

The input given at runtime, does not match the defined input type of the InternalPipelineModule

exception evalquiz_proto.shared.exceptions.ResultException

Bases: PipelineExecutionException

The generated result by the language model is not able to be further processed, due to incompatibilities.

exception evalquiz_proto.shared.exceptions.ResultSectionNotFoundException

Bases: ResultException

The result section, marked with <result></result> tags was not found in the given text.

exception evalquiz_proto.shared.exceptions.ResultSectionNotParsableException

Bases: ResultException

The structure of the result section, marked with <result></result> tags does not match the QuestionType.

evalquiz_proto.shared.internal_lecture_material module

class evalquiz_proto.shared.internal_lecture_material.InternalLectureMaterial(local_path: Path, lecture_material: LectureMaterial)

Bases: LectureMaterial

A lecture material with an additional local path pointing to the file.

cast_to_lecture_material() → LectureMaterial

Casts self object to object of superclass: LectureMaterial. This method is required, as Python does not feature object type casting.

Returns:

Self object casted into LectureMaterial.

Return type:

LectureMaterial

classmethod from_mongodb_document(document: dict[str, Any]) → InternalLectureMaterial

Constructor of self from pymongo representation.

Parameters:

document (dict[str, Any]) – Dictionary containing hash and serialized self.

local_path: Path = PosixPath('.')

to_mongodb_document() → dict[str, Any]

Encodes self to a representation that can be inserted by pymongo.

Returns:

Dictionary containing hash and serialized self.

Return type:

dict[str, Any]

update_hash(rename_file: bool = False) → None

Updates LectureMaterial hash with file contents.

Parameters:

rename_file (bool, optional) – Filename of referenced file is changed to the updated hash, if set to True. Defaults to False.

update_mimetype() → None

Evaluates if given mimetype matches mimetype of file at local_path. Sets mimetype to new value, if mimetype does not match mimetype of file at local_path.

Raises:

MimetypeNotDetectedException –

verify_hash(other_hash: str | None = None) → bool

Verifies that the file behind local_path is consistent with other_hash.

Parameters:

other_hash – An optional hash parameter to test consistency against. If other_hash is None, self.hash is used as other_hash.

Returns:

True, if other_hash matches calculated hash of file.

evalquiz_proto.shared.mimetype_resolver module

class evalquiz_proto.shared.mimetype_resolver.MimetypeResolver

Bases: object

static fixed_guess_extension(mimetype: str) → str | None

Fixes the missing markdown type in Python’s mimetypes library.

Parameters:

mimetype (str) – Mimetype to guess extension for.

Returns:

Extension with leading dot.

Return type:

Optional[str]

static fixed_guess_type(suffix: str) → str | None

Fixes the missing markdown type in Python’s mimetypes library.

Parameters:

local_path (Path) – Path to guess type for.

Returns:

The guessed mime type.

Return type:

str

Raises:

MimetypeNotDetectedException –

evalquiz_proto.shared.path_dictionary_controller module

class evalquiz_proto.shared.path_dictionary_controller.PathDictionaryController(mongodb_client: MongoClient[dict[str, Any]], mongodb_database: str = 'local_path_db')

Bases: object

The PathDictionaryController manages str aliases to paths for local files. Multiple PathDictionaryController instances are able to share their state using MongoDB.

async add_file_async(local_path: Path, hash: str, binary_iterator: AsyncIterator[bytes], overwrite: bool = True, name: str = '') → None

A new file is created asynchronously at the specified location from a stream. System operations are carried out async to allow large file sizes and reduce the memory footprint.

Parameters:

• local_path – The system path to the location where the file is created.

• hash – Hash to reference the file.

• binary_iterator – Yields binary data of the file itself.

• overwrite – Boolean to describe if an existing file can be overwritten.

• name – Name or description of the file to add.

Raises:

FileOverwriteNotPermittedException –

copy_and_load_file(source_local_path: Path, destination_local_path: Path, hash: str, name: str = '') → None

Copies file to destination and loads the destination file with self.load_file(…).

Parameters:

• source_local_path (Path) – Source file local path.

• destination_local_path (Path) – Destination file local path.

• hash (str) – Hash to reference the file.

delete_file(hash: str) → None

Deletes the reference to the file and the file itself from the filesystem.

Parameters:

local_path – The system path to the file.

async get_file_from_hash_async(hash: str, content_partition_size: int = 500000000) → tuple[str, AsyncIterator[bytes]]

Streams a local file using the given hash and returns its mimetype.

Parameters:

• hash (str) – Hash to reference the file.

• content_partition_size (int, optional) – The maximum filesize in bytes that a packet can have. Defaults to 5*10**8.

Raises:

KeyError – If file is not found under the given hash.

Returns:

A tuple with the mimetype at the first index and the asynchronous iterator for streaming at the second index.

Return type:

tuple[str, AsyncIterator[MaterialUploadData]]

get_file_path_from_hash(hash: str) → Path

Retrieves local path from hash.

Parameters:

hash (str) – Hash to reference the file.

Raises:

KeyError – If file is not found under the given hash.

Returns:

The path to the local file.

Return type:

Path

get_material_hashes() → list[str]

Retrieves all hashes of the internally referenced files.

Returns:

A set of strings

get_material_name(hash: str) → str

Returns material name from hash.

Parameters:

hash (str) – Hash to reference the file.

Raises:

KeyError – If file is not found under the given hash

Returns:

Material name.

Return type:

str

load_file(local_path: Path, hash: str, name: str = '') → None

Adds a file to the internal pool of files.

Parameters:

• local_path (Path) – The system path to the file.

• hash (str) – Hash to reference the file.

• name – Name or description of the file to load.

unload_material(hash: str) → None

Removes the internal representation of a lecture material. Does not delete the file.

Parameters:

hash – Hash to reference the file.

Module contents