Commit Graph

4 Commits

Author SHA1 Message Date
Henry Ruhs
4ccf4c24c7
Next (#477)
* Add real_hatgan_x4 model

* Mark it as NEXT

* Force download to be executed and exit

* Fix frame per second interpolation

* 5 to 68 landmark (#456)

* changes

* changes

* Adjust model url

* Cleanup 5 to 68 landmark convertion

* Move everything to face analyser

* Introduce matrix only face helper

* Revert facefusion.ini

* Adjust limit due false positive analysis

* changes (#457)

* Use pixel format yuv422p to merge video

* Fix some code

* Minor cleanup

* Add gpen_bfr_1024 and gpen_bfr_2048

* Revert it back to yuv420p due compatibility issues

* Add debug back to ffmpeg

* Add debug back to ffmpeg

* Migrate to conda (#461)

* Migrate from venv to conda

* Migrate from venv to conda

* Message when conda is not activated

* Use release for every slider (#463)

* Use release event handler for every slider

* Move more sliders to release handler

* Move more sliders to release handler

* Add get_ui_components() to simplify code

* Revert some changes on frame slider

* Add the first iteration of a frame colorizer

* Support for the DDColor model

* Improve model file handling

* Improve model file handling part2

* Remove deoldify

* Remove deoldify

* Voice separator (#468)

* changes

* changes

* changes

* changes

* changes

* changes

* Rename audio extractor to voice extractor

* Cosmetic changes

* Cosmetic changes

* Fix fps lowering and boosting

* Fix fps lowering and boosting

* Fix fps lowering and boosting

* Some refactoring for audio.py and some astype() here and there (#470)

* Some refactoring for audio.py and some astype() here and there

* Fix lint

* Spacing

* Add mp3 to benchmark suite for lip syncer testing

* Improve naming

* Adjust chunk size

* Use higher quality

* Revert "Use higher quality"

This reverts commit d32f287572.

* Improve naming in ffmpeg.py

* Simplify code

* Better fps calculation

* Fix naming here and there

* Add back real esrgan x2

* Remove trailing comma

* Update wording and README

* Use semaphore to prevent frame colorizer memory issues

* Revert "Remove deoldify"

This reverts commit bd8034cbc7.

* Remove unused type from frame colorizer

* Adjust naming

* Add missing clear of model initializer

* Change nvenc preset mappping to support old FFMPEG 4

* Update onnxruntime to 1.17.1

* Fix lint

* Prepare 2.5.0

* Fix Gradio overrides

* Add Deoldify Artistic back

* Feat/audio refactoring (#476)

* Improve audio naming and variables

* Improve audio naming and variables

* Refactor voice extractor like crazy

* Refactor voice extractor like crazy

* Remove spaces

* Update the usage

---------

Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com>
2024-04-09 15:40:55 +02:00
Henry Ruhs
c77493ff9a
Next (#384)
* feat/yoloface (#334)

* added yolov8 to face_detector (#323)

* added yolov8 to face_detector

* added yolov8 to face_detector

* Initial cleanup and renaming

* Update README

* refactored detect_with_yoloface (#329)

* refactored detect_with_yoloface

* apply review

* Change order again

* Restore working code

* modified code (#330)

* refactored detect_with_yoloface

* apply review

* use temp_frame in detect_with_yoloface

* reorder

* modified

* reorder models

* Tiny cleanup

---------

Co-authored-by: tamoharu <133945583+tamoharu@users.noreply.github.com>

* include audio file functions (#336)

* Add testing for audio handlers

* Change order

* Fix naming

* Use correct typing in choices

* Update help message for arguments, Notation based wording approach (#347)

* Update help message for arguments, Notation based wording approach

* Fix installer

* Audio functions (#345)

* Update ffmpeg.py

* Create audio.py

* Update ffmpeg.py

* Update audio.py

* Update audio.py

* Update typing.py

* Update ffmpeg.py

* Update audio.py

* Rename Frame to VisionFrame (#346)

* Minor tidy up

* Introduce audio testing

* Add more todo for testing

* Add more todo for testing

* Fix indent

* Enable venv on the fly

* Enable venv on the fly

* Revert venv on the fly

* Revert venv on the fly

* Force Gradio to shut up

* Force Gradio to shut up

* Clear temp before processing

* Reduce terminal output

* include audio file functions

* Enforce output resolution on merge video

* Minor cleanups

* Add age and gender to face debugger items (#353)

* Add age and gender to face debugger items

* Rename like suggested in the code review

* Fix the output framerate vs. time

* Lip Sync (#356)

* Cli implementation of wav2lip

* - create get_first_item()
- remove non gan wav2lip model
- implement video memory strategy
- implement get_reference_frame()
- implement process_image()
- rearrange crop_mask_list
- implement test_cli

* Simplify testing

* Rename to lip syncer

* Fix testing

* Fix testing

* Minor cleanup

* Cuda 12 installer (#362)

* Make cuda nightly (12) the default

* Better keep legacy cuda just in case

* Use CUDA and ROCM versions

* Remove MacOS options from installer (CoreML include in default package)

* Add lip-syncer support to source component

* Add lip-syncer support to source component

* Fix the check in the source component

* Add target image check

* Introduce more helpers to suite the lip-syncer needs

* Downgrade onnxruntime as of buggy 1.17.0 release

* Revert "Downgrade onnxruntime as of buggy 1.17.0 release"

This reverts commit f4a7ae6824.

* More testing and add todos

* Fix the frame processor API to at least not throw errors

* Introduce dict based frame processor inputs (#364)

* Introduce dict based frame processor inputs

* Forgot to adjust webcam

* create path payloads (#365)

* create index payload to paths for process_frames

* rename to payload_paths

* This code now is poetry

* Fix the terminal output

* Make lip-syncer work in the preview

* Remove face debugger test for now

* Reoder reference_faces, Fix testing

* Use inswapper_128 on buggy onnxruntime 1.17.0

* Undo inswapper_128_fp16 duo broken onnxruntime 1.17.0

* Undo inswapper_128_fp16 duo broken onnxruntime 1.17.0

* Fix lip_syncer occluder & region mask issue

* Fix preview once in case there was no output video fps

* fix lip_syncer custom fps

* remove unused import

* Add 68 landmark functions (#367)

* Add 68 landmark model

* Add landmark to face object

* Re-arrange and modify typing

* Rename function

* Rearrange

* Rearrange

* ignore type

* ignore type

* change type

* ignore

* name

* Some cleanup

* Some cleanup

* Opps, I broke something

* Feat/face analyser refactoring (#369)

* Restructure face analyser and start TDD

* YoloFace and Yunet testing are passing

* Remove offset from yoloface detection

* Cleanup code

* Tiny fix

* Fix get_many_faces()

* Tiny fix (again)

* Use 320x320 fallback for retinaface

* Fix merging mashup

* Upload wave2lip model

* Upload 2dfan2 model and rename internal to face_predictor

* Downgrade onnxruntime for most cases

* Update for the face debugger to render landmark 68

* Try to make detect_face_landmark_68() and detect_gender_age() more uniform

* Enable retinaface testing for 320x320

* Make detect_face_landmark_68() and detect_gender_age() as uniform as … (#370)

* Make detect_face_landmark_68() and detect_gender_age() as uniform as possible

* Revert landmark scale and translation

* Make box-mask for lip-syncer adjustable

* Add create_bbox_from_landmark()

* Remove currently unused code

* Feat/uniface (#375)

* add uniface (#373)

* Finalize UniFace implementation

---------

Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com>

* My approach how todo it

* edit

* edit

* replace vertical blur with gaussian

* remove region mask

* Rebase against next and restore method

* Minor improvements

* Minor improvements

* rename & add forehead padding

* Adjust and host uniface model

* Use 2dfan4 model

* Rename to face landmarker

* Feat/replace bbox with bounding box (#380)

* Add landmark 68 to 5 convertion

* Add landmark 68 to 5 convertion

* Keep 5, 5/68 and 68 landmarks

* Replace kps with landmark

* Replace bbox with bounding box

* Reshape face_landmark5_list different

* Make yoloface the default

* Move convert_face_landmark_68_to_5 to face_helper

* Minor spacing issue

* Dynamic detector sizes according to model (#382)

* Dynamic detector sizes according to model

* Dynamic detector sizes according to model

* Undo false commited files

* Add lib syncer model to the UI

* fix halo (#383)

* Bump to 2.3.0

* Update README and wording

* Update README and wording

* Fix spacing

* Apply _vision suffix

* Apply _vision suffix

* Apply _vision suffix

* Apply _vision suffix

* Apply _vision suffix

* Apply _vision suffix

* Apply _vision suffix, Move mouth mask to face_masker.py

* Apply _vision suffix

* Apply _vision suffix

* increase forehead padding

---------

Co-authored-by: tamoharu <133945583+tamoharu@users.noreply.github.com>
Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com>
2024-02-14 14:08:29 +01:00
Henry Ruhs
a6809c3ccb
Next (#144)
* Improve typing for our callbacks

* Return 0 for get_download_size

* Introduce ONNX powered face enhancer

* Introduce ONNX powered face enhancer

* Introduce ONNX powered face enhancer

* Remove tile processing from frame enhancer

* Fix video compress translation for libvpx-vp9

* Allow zero values for video compression

* Develop (#134)

* Introduce model options to the frame processors

* Finish UI to select frame processors models

* Simplify frame processors options

* Fix lint in CI

* Rename all kind of settings to options

* Add blend to enhancers

* Simplify webcam mode naming

* Bypass SSL issues under Windows

* Fix blend of frame enhancer

* Massive CLI refactoring, Register and apply ARGS via the frame processors

* Refine UI theme and introduce donate button

* Update dependencies and fix cpu only torch

* Update dependencies and fix cpu only torch

* Fix theme, Fix frame_processors in headless mode

* Remove useless astype

* Disable CoreML for the ONNX face enhancer

* Disable CoreML for the ONNX face enhancer

* Predict webcam too

* Improve resize of preview

* Change output quality defaults, Move options to the right

* Support for codeformer model

* Update the typo

* Add GPEN and GFPGAN 1.2

* Extract blend_frame methods

* Extend the installer

* Revert broken Gradio

* Rework on ui components

* Move output path selector to the output options

* Remove tons of pointless component updates

* Reset more base theme styling

* Use latest Gradio

* Fix the sliders

* More styles

* Update torch to 2.1.0

* Add RealESRNet_x4plus

* Fix that button

* Use latest onnxruntime-silicon

* Looks stable to me

* Lowercase model keys, Update preview and readme
2023-10-09 10:16:13 +02:00
Henry Ruhs
82eaf76da8
Develop (#73)
* Cosmetic changes

* Cosmetic changes

* Run single warm up for the benchmark suite

* Use latest version of Gradio

* More testing

* Introduce basic installer

* Fix typo

* Move more to installer file

* Fix the installer with the uninstall all trick

* Adjust wording

* Fix coreml in installer

* Allow Pyhton 3.9

* Add VENV to installer

* Just some cosmetics

* Just some cosmetics

* Dedicated headless mode, Refine API of UI layouts

* Use --headless for pytest

* Fix testing for Windows

* Normalize output path that lacks extension

* Fix CI for Windows

* Fix CI for Windows

* UI to change output path

* Add conda support for the installer

* Improve installer quite a bit

* Drop conda support

* Install community wheels for coreml silicon

* Improve output video component

* Fix silicon wheel downloading

* Remove venv from installer as we cannot activate via subprocess

* Use join to create wheel name

* Refine the output path normalization

* Refine the output path normalization

* Introduce ProcessMode and rename some methods

* Introduce ProcessMode and rename some methods

* Basic webcam integration and open_ffmpeg()

* Basic webcam integration part2

* Benchmark resolutions now selectable

* Rename benchmark resolution back to benchmark runs

* Fix repeating output path in UI

* Keep output_path untouched if not resolvable

* Add more cases to normalize output path

* None for those tests that don't take source path into account

* Finish basic webcam integration, UI layout now with custom run()

* Fix CI and hide link in webcam UI

* Cosmetics on webcam UI

* Move get_device to utilities

* Fix CI

* Introduce output-image-quality, Show and hide UI according to target media type

* Benchmark with partial result updates

* fix: trim frame sliders not appearing after draggin video

* fix: output and temp frame setting inputs not appearing

* Fix: set increased update delay to 250ms to let Gradio update conditional inputs properly

* Reverted .gitignore

* Adjust timings

* Remove timeout hacks and get fully event driven

* Update dependencies

* Update dependencies

* Revert NSFW library, Conditional unset trim args

* Face selector works better on preview slider release

* Add limit resources to UI

* Introduce vision.py for all CV2 operations, Rename some methods

* Add restoring audio failed

* Decouple updates for preview image and preview frame slider, Move reduce_preview_frame to vision

* Refactor detect_fps based on JSON output

* Only webcam when open

* More conditions to vision.py

* Add udp and v4l2 streaming to webcam UI

* Detect v4l2 device to be used

* Refactor code a bit

* Use static max memory for UI

* Fix CI

* Looks stable to me

* Update preview

* Update preview

---------

Co-authored-by: Sumit <vizsumit@gmail.com>
2023-09-06 00:25:18 +02:00