refactor: assemble inference engines through a shared composition root
- add build_engine() to astrai.inference.engine as the single load-place-wire path for InferenceEngine, accepting a checkpoint path or live model/tokenizer plus passthrough engine kwargs - migrate the server lifespan, generate CLI, humaneval/ifeval evals, and all three demos to build_engine; app._create_engine collapses into a direct call - export build_engine from astrai and astrai.inference - parameterize the autoregressive demo with --prompt one-shot continuation plus model path and sampling knobs, exiting cleanly on !exit or EOF - cover the composition root with unit tests for live-object assembly, kwargs passthrough, and argument validation
This commit is contained in:
+2
-1
@@ -17,7 +17,7 @@ from astrai.dataset import (
|
||||
StoreFactory,
|
||||
)
|
||||
from astrai.factory import BaseFactory
|
||||
from astrai.inference import InferenceEngine, get_app, run_server, sample
|
||||
from astrai.inference import InferenceEngine, build_engine, get_app, run_server, sample
|
||||
from astrai.inference.network import ProtocolHandler
|
||||
from astrai.inference.runtime.sample import SamplingPipeline
|
||||
from astrai.logging import setup_logging
|
||||
@@ -67,6 +67,7 @@ __all__ = [
|
||||
"EncoderConfig",
|
||||
"ExecutorFactory",
|
||||
"InferenceEngine",
|
||||
"build_engine",
|
||||
"LoRAConfig",
|
||||
"Pipeline",
|
||||
"PipelineConfig",
|
||||
|
||||
Reference in New Issue
Block a user