facefusion/facefusion/processors/typing.py
Henry Ruhs 7a09479fb5
3.1.0 (#839)
* Replace audio whenever set via source

* add H264_qsv&HEVC_qsv (#768)

* Update ffmpeg.py

* Update choices.py

* Update typing.py

* Fix spaces and newlines

* Fix return type

* Introduce hififace swapper

* Disable stream for expression restorer

* Webcam polishing part1 (#796)

* Cosmetics on ignore comments

* Testing for replace audio

* Testing for restore audio

* Testing for restore audio

* Fix replace_audio()

* Remove shortest and use fixed video duration

* Remove shortest and use fixed video duration

* Prevent duplicate entries to local PATH

* Do hard exit on invalid args

* Need for Python 3.10

* Fix state of face selector

* Fix OpenVINO by aliasing GPU.0 to GPU

* Fix OpenVINO by aliasing GPU.0 to GPU

* Fix/age modifier styleganex 512 (#798)

* fix

* styleganex template

* changes

* changes

* fix occlusion mask

* add age modifier scale

* change

* change

* hardcode

* Cleanup

* Use model_sizes and model_templates variables

* No need for prepare when just 2 lines of code

* Someone used spaces over tabs

* Revert back [0][0]

---------

Co-authored-by: harisreedhar <h4harisreedhar.s.s@gmail.com>

* Feat/update gradio5 (#799)

* Update to Gradio 5

* Remove overrides for Gradio

* Fix dark mode for Gradio

* Polish errors

* More styles for tabs and co

* Make slider inputs and reset like a unit

* Make slider inputs and reset like a unit

* Adjust naming

* Improved color matching (#800)

* aura fix

* fix import

* move to vision.py

* changes

* changes

* changes

* changes

* further reduction

* add test

* better test

* change name

* Minor cleanup

* Minor cleanup

* Minor cleanup

* changes (#801)

* Switch to official assets repo

* Add __pycache__ to gitignore

* Gradio pinned python-multipart to 0.0.12

* Update dependencies

* Feat/temp path second try (#802)

* Terminate base directory from temp helper

* Partial adjust program codebase

* Move arguments around

* Make `-j` absolete

* Resolve args

* Fix job register keys

* Adjust date test

* Finalize temp path

* Update onnxruntime

* Update dependencies

* Adjust color for checkboxes

* Revert due terrible performance

* Fix/enforce vp9 for webm (#805)

* Simple fix to enforce vp9 for webm

* Remove suggest methods from program helper

* Cleanup ffmpeg.py a bit

* Update onnxruntime (second try)

* Update onnxruntime (second try)

* Remove cudnn_conv_algo_search tweaks

* Remove cudnn_conv_algo_search tweaks

* changes

* add both mask instead of multiply

* adaptive color correction

* changes

* remove model size requirement

* changes

* add to facefusion.ini

* changes

* changes

* changes

* Add namespace for dfm creators

* Release five frame enhancer models

* Remove vendor from model name

* Remove vendor from model name

* changes

* changes

* changes

* changes

* Feat/download providers (#809)

* Introduce download providers

* update processors download method

* add ui

* Fix CI

* Adjust UI component order, Use download resolver for benchmark

* Remove is_download_done()

* Introduce download provider set, Remove choices method from execution, cast all dict keys() via list()

* Fix spacing

---------

Co-authored-by: harisreedhar <h4harisreedhar.s.s@gmail.com>

* Fix model paths for 3.1.0

* Introduce bulk-run (#810)

* Introduce bulk-run

* Make bulk run bullet proof

* Integration test for bulk-run

* new alignment

* Add safer global named resolve_file_pattern() (#811)

* Allow bulk runner with target pattern only

* changes

* changes

* Update Python to 3.12 for CI (#813)

* changes

* Improve NVIDIA device lookups

* Rename template key to deepfacelive

* Fix name

* Improve resolve download

* Rename bulk-run to batch-run

* Make deep swapper inputs universal

* Add more deepfacelive models

* Use different morph value

* Feat/simplify hashes sources download (#814)

* Extract download directory path from assets path

* Fix lint

* Fix force-download command, Fix urls in frame enhancer

* changes

* fix warp_face_by_bounding_box dtype error

* DFM Morph (#816)

* changes

* Improve wording, Replace [None], SideQuest: clean forward() of age modifier

* SideQuest: clean forward() of face enhancer

---------

Co-authored-by: henryruhs <info@henryruhs.com>

* Fix preview refresh after slide

* Add more deepfacelive models (#817)

* Add more deepfacelive models

* Add more deepfacelive models

* Fix deep swapper sizes

* Kill accent colors, Number input styles for Chrome

* Simplify thumbnail-item looks

* Fix first black screen

* Introduce model helper

* ci.yml: Add macOS on ARM64 to the testing (#818)

* ci.yml: Add macOS on ARM64 to the testing

* ci.yml: uses: AnimMouse/setup-ffmpeg@v1

* ci.yml: strategy: matrix: os: macos-latest,

* - name: Set up FFmpeg

* Update .github/workflows/ci.yml

* Update ci.yml

---------

Co-authored-by: Henry Ruhs <info@henryruhs.com>

* Show/hide morph slider for deep swapper (#822)

* remove dfl_head and update dfl_whole_face template

* Add deep swapper models by Mats

* Add deep swapper models by Druuzil

* Add deep swapper models by Rumateus

* Implement face enhancer weight for codeformer, Side Quest: has proces… (#823)

* Implement face enhancer weight for codeformer, Side Quest: has processor checks

* Fix typo

* Fix face enhancer blend in UI

* Use static model set creation

* Add deep swapper models by Jen

* Introduce create_static_model_set() everywhere (#824)

* Move clear over to the UI (#825)

* Fix model key

* Undo restore_audio()

* Switch to latest XSeg

* Switch to latest XSeg

* Switch to latest XSeg

* Use resolve_download_url() everywhere, Vanish --skip-download flag

* Fix resolve_download_url

* Fix space

* Kill resolve_execution_provider_keys() and move fallbacks where they belong

* Kill resolve_execution_provider_keys() and move fallbacks where they belong

* Remove as this does not work

* Change TempFrameFormat order

* Fix CoreML partially

* Remove duplicates (Rumateus is the creator)

* Add deep swapper models by Edel

* Introduce download scopes (#826)

* Introduce download scopes

* Limit download scopes to force-download command

* Change source-paths behaviour

* Fix space

* Update README

* Rename create_log_level_program to create_misc_program

* Fix wording

* Fix wording

* Update dependencies

* Use tolerant for video_memory_strategy in benchmark

* Feat/ffmpeg with progress (#827)

* FFmpeg with progress bar

* Fix typing

* FFmpeg with progress bar part2

* Restore streaming wording

* Change order in choices and typing

* Introduce File using list_directory() (#830)

* Feat/local deep swapper models (#832)

* Local model support for deep swapper

* Local model support for deep swapper part2

* Local model support for deep swapper part3

* Update yet another dfm by Druuzil

* Refactor/choices and naming (#833)

* Refactor choices, imports and naming

* Refactor choices, imports and naming

* Fix styles for tabs, Restore toast

* Update yet another dfm by Druuzil

* Feat/face masker models (#834)

* Introduce face masker models

* Introduce face masker models

* Introduce face masker models

* Register needed step keys

* Provide different XSeg models

* Simplify model context

* Fix out of range for trim frame, Fix ffmpeg extraction count (#836)

* Fix out of range for trim frame, Fix ffmpeg extraction count

* Move restrict of trim frame to the core, Make sure all values are within the range

* Fix and merge testing

* Fix typing

* Add region mask for deep swapper

* Adjust wording

* Move FACE_MASK_REGIONS to choices

* Update dependencies

* Feat/download provider fallback (#837)

* Introduce download providers fallback, Use CURL everywhre

* Fix CI

* Use readlines() over readline() to avoid while

* Use readlines() over readline() to avoid while

* Use readlines() over readline() to avoid while

* Use communicate() over wait()

* Minor updates for testing

* Stop webcam on source image change

* Feat/webcam improvements (#838)

* Detect available webcams

* Fix CI, Move webcam id dropdown to the sidebar, Disable warnings

* Fix CI

* Remove signal on hard_exit() to prevent exceptions

* Fix border color in toast timer

* Prepare release

* Update preview

* Update preview

* Hotfix progress bar

---------

Co-authored-by: DDXDB <38449595+DDXDB@users.noreply.github.com>
Co-authored-by: harisreedhar <h4harisreedhar.s.s@gmail.com>
Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com>
Co-authored-by: Christian Clauss <cclauss@me.com>
2024-12-24 12:46:56 +01:00

158 lines
5.4 KiB
Python

from typing import Any, Dict, List, Literal, TypedDict
from numpy._typing import NDArray
from facefusion.typing import AppContext, AudioFrame, Face, FaceSet, VisionFrame
AgeModifierModel = Literal['styleganex_age']
DeepSwapperModel = str
ExpressionRestorerModel = Literal['live_portrait']
FaceDebuggerItem = Literal['bounding-box', 'face-landmark-5', 'face-landmark-5/68', 'face-landmark-68', 'face-landmark-68/5', 'face-mask', 'face-detector-score', 'face-landmarker-score', 'age', 'gender', 'race']
FaceEditorModel = Literal['live_portrait']
FaceEnhancerModel = Literal['codeformer', 'gfpgan_1.2', 'gfpgan_1.3', 'gfpgan_1.4', 'gpen_bfr_256', 'gpen_bfr_512', 'gpen_bfr_1024', 'gpen_bfr_2048', 'restoreformer_plus_plus']
FaceSwapperModel = Literal['blendswap_256', 'ghost_1_256', 'ghost_2_256', 'ghost_3_256', 'hififace_unofficial_256', 'inswapper_128', 'inswapper_128_fp16', 'simswap_256', 'simswap_unofficial_512', 'uniface_256']
FrameColorizerModel = Literal['ddcolor', 'ddcolor_artistic', 'deoldify', 'deoldify_artistic', 'deoldify_stable']
FrameEnhancerModel = Literal['clear_reality_x4', 'lsdir_x4', 'nomos8k_sc_x4', 'real_esrgan_x2', 'real_esrgan_x2_fp16', 'real_esrgan_x4', 'real_esrgan_x4_fp16', 'real_esrgan_x8', 'real_esrgan_x8_fp16', 'real_hatgan_x4', 'real_web_photo_x4', 'realistic_rescaler_x4', 'remacri_x4', 'siax_x4', 'span_kendata_x4', 'swin2_sr_x4', 'ultra_sharp_x4']
LipSyncerModel = Literal['wav2lip_96', 'wav2lip_gan_96']
FaceSwapperSet = Dict[FaceSwapperModel, List[str]]
AgeModifierInputs = TypedDict('AgeModifierInputs',
{
'reference_faces' : FaceSet,
'target_vision_frame' : VisionFrame
})
DeepSwapperInputs = TypedDict('DeepSwapperInputs',
{
'reference_faces' : FaceSet,
'target_vision_frame' : VisionFrame
})
ExpressionRestorerInputs = TypedDict('ExpressionRestorerInputs',
{
'reference_faces' : FaceSet,
'source_vision_frame' : VisionFrame,
'target_vision_frame' : VisionFrame
})
FaceDebuggerInputs = TypedDict('FaceDebuggerInputs',
{
'reference_faces' : FaceSet,
'target_vision_frame' : VisionFrame
})
FaceEditorInputs = TypedDict('FaceEditorInputs',
{
'reference_faces' : FaceSet,
'target_vision_frame' : VisionFrame
})
FaceEnhancerInputs = TypedDict('FaceEnhancerInputs',
{
'reference_faces' : FaceSet,
'target_vision_frame' : VisionFrame
})
FaceSwapperInputs = TypedDict('FaceSwapperInputs',
{
'reference_faces' : FaceSet,
'source_face' : Face,
'target_vision_frame' : VisionFrame
})
FrameColorizerInputs = TypedDict('FrameColorizerInputs',
{
'target_vision_frame' : VisionFrame
})
FrameEnhancerInputs = TypedDict('FrameEnhancerInputs',
{
'target_vision_frame' : VisionFrame
})
LipSyncerInputs = TypedDict('LipSyncerInputs',
{
'reference_faces' : FaceSet,
'source_audio_frame' : AudioFrame,
'target_vision_frame' : VisionFrame
})
ProcessorStateKey = Literal\
[
'age_modifier_model',
'age_modifier_direction',
'deep_swapper_model',
'deep_swapper_morph',
'expression_restorer_model',
'expression_restorer_factor',
'face_debugger_items',
'face_editor_model',
'face_editor_eyebrow_direction',
'face_editor_eye_gaze_horizontal',
'face_editor_eye_gaze_vertical',
'face_editor_eye_open_ratio',
'face_editor_lip_open_ratio',
'face_editor_mouth_grim',
'face_editor_mouth_pout',
'face_editor_mouth_purse',
'face_editor_mouth_smile',
'face_editor_mouth_position_horizontal',
'face_editor_mouth_position_vertical',
'face_editor_head_pitch',
'face_editor_head_yaw',
'face_editor_head_roll',
'face_enhancer_model',
'face_enhancer_blend',
'face_enhancer_weight',
'face_swapper_model',
'face_swapper_pixel_boost',
'frame_colorizer_model',
'frame_colorizer_size',
'frame_colorizer_blend',
'frame_enhancer_model',
'frame_enhancer_blend',
'lip_syncer_model'
]
ProcessorState = TypedDict('ProcessorState',
{
'age_modifier_model' : AgeModifierModel,
'age_modifier_direction' : int,
'deep_swapper_model' : DeepSwapperModel,
'deep_swapper_morph' : int,
'expression_restorer_model' : ExpressionRestorerModel,
'expression_restorer_factor' : int,
'face_debugger_items' : List[FaceDebuggerItem],
'face_editor_model' : FaceEditorModel,
'face_editor_eyebrow_direction' : float,
'face_editor_eye_gaze_horizontal' : float,
'face_editor_eye_gaze_vertical' : float,
'face_editor_eye_open_ratio' : float,
'face_editor_lip_open_ratio' : float,
'face_editor_mouth_grim' : float,
'face_editor_mouth_pout' : float,
'face_editor_mouth_purse' : float,
'face_editor_mouth_smile' : float,
'face_editor_mouth_position_horizontal' : float,
'face_editor_mouth_position_vertical' : float,
'face_editor_head_pitch' : float,
'face_editor_head_yaw' : float,
'face_editor_head_roll' : float,
'face_enhancer_model' : FaceEnhancerModel,
'face_enhancer_blend' : int,
'face_enhancer_weight' : float,
'face_swapper_model' : FaceSwapperModel,
'face_swapper_pixel_boost' : str,
'frame_colorizer_model' : FrameColorizerModel,
'frame_colorizer_size' : str,
'frame_colorizer_blend' : int,
'frame_enhancer_model' : FrameEnhancerModel,
'frame_enhancer_blend' : int,
'lip_syncer_model' : LipSyncerModel
})
ProcessorStateSet = Dict[AppContext, ProcessorState]
AgeModifierDirection = NDArray[Any]
DeepSwapperMorph = NDArray[Any]
FaceEnhancerWeight = NDArray[Any]
LivePortraitPitch = float
LivePortraitYaw = float
LivePortraitRoll = float
LivePortraitExpression = NDArray[Any]
LivePortraitFeatureVolume = NDArray[Any]
LivePortraitMotionPoints = NDArray[Any]
LivePortraitRotation = NDArray[Any]
LivePortraitScale = NDArray[Any]
LivePortraitTranslation = NDArray[Any]