Ryan Dick
477d87ec31
Fix layer patch dtype selection for CLIP text encoder models.
2024-12-29 21:48:51 +00:00
Ryan Dick
8b4b0ff0cf
Fix bug in CustomConv1d and CustomConv2d patch calculations.
2024-12-29 19:10:19 +00:00
Ryan Dick
6fd9b0a274
Delete old sidecar wrapper implementation. This functionality has moved into the custom layers.
2024-12-29 17:33:08 +00:00
Ryan Dick
a8bef59699
First pass at making custom layer patches work with weights streamed from the CPU to the GPU.
2024-12-29 17:01:37 +00:00
Ryan Dick
6d49ee839c
Switch the LayerPatcher to use 'custom modules' to manage layer patching.
2024-12-29 01:18:30 +00:00
Ryan Dick
0525f967c2
Fix the _autocast_forward_with_patches() function for CustomConv1d and CustomConv2d.
2024-12-29 00:22:37 +00:00
Ryan Dick
2855bb6b41
Update BaseLayerPatch.get_parameters(...) to accept a dict of orig_parameters rather than orig_module. This will enable compatibility between patching and cpu->gpu streaming.
2024-12-28 21:12:53 +00:00
Ryan Dick
20acfc9a00
Raise in CustomEmbedding and CustomGroupNorm if a patch is applied.
2024-12-28 20:49:17 +00:00
Ryan Dick
918f541af8
Add unit test for a SetParameterLayer patch applied to a CustomFluxRMSNorm layer.
2024-12-28 20:44:48 +00:00
Ryan Dick
93e76b61d6
Add CustomFluxRMSNorm layer.
2024-12-28 20:33:38 +00:00
Ryan Dick
f692e217ea
Add patch support to CustomConv1d and CustomConv2d (no unit tests yet).
2024-12-27 22:23:17 +00:00
Ryan Dick
f2981979f9
Get custom layer patches working with all quantized linear layer types.
2024-12-27 22:00:22 +00:00
Ryan Dick
ef970a1cdc
Add support for FluxControlLoRALayer in CustomLinear layers and add a unit test for it.
2024-12-27 21:00:47 +00:00
Ryan Dick
e24e386a27
Add support for patches to CustomModuleMixin and add a single unit test (more to come).
2024-12-27 18:57:13 +00:00
Ryan Dick
b06d61e3c0
Improve custom layer wrap/unwrap logic.
2024-12-27 16:29:48 +00:00
Ryan Dick
7d6ab0ceb2
Add a CustomModuleMixin class with a flag for enabling/disabling autocasting (since it incurs some runtime speed overhead.)
2024-12-26 20:08:30 +00:00
Ryan Dick
a8b2c4c3d2
Add inference tests for all custom module types (i.e. to test autocasting from cpu to device).
2024-12-26 18:33:46 +00:00
Ryan Dick
987c9ae076
Move custom autocast modules to separate files in a custom_modules/ directory.
2024-12-24 22:21:31 +00:00
Ryan Dick
6d7314ac0a
Consolidate the LayerPatching patching modes into a single implementation.
2024-12-24 15:57:54 +00:00
Ryan Dick
80db9537ff
Rename model_patcher.py -> layer_patcher.py.
2024-12-24 15:57:54 +00:00
Ryan Dick
6f926f05b0
Update apply_smart_model_patches() so that layer restore matches the behavior of non-smart mode.
2024-12-24 15:57:54 +00:00
Ryan Dick
61253b91f1
Enable LoRAPatcher.apply_smart_lora_patches(...) throughout the stack.
2024-12-24 15:57:54 +00:00
Ryan Dick
cefcb340d9
Add LoRAPatcher.smart_apply_lora_patches()
2024-12-24 15:57:54 +00:00
Ryan Dick
7214d4969b
Workaround a weird quirk of QuantState.to() and add a unit test to exercise it.
2024-12-24 14:32:11 +00:00
Ryan Dick
f8a6accf8a
Fix bitsandbytes imports to avoid ImportErrors on MacOS.
2024-12-24 14:32:11 +00:00
Ryan Dick
f8ab414f99
Add CachedModelOnlyFullLoad to mirror the CachedModelWithPartialLoad for models that cannot or should not be partially loaded.
2024-12-24 14:32:11 +00:00
Ryan Dick
c6795a1b47
Make CachedModelWithPartialLoad work with models that have non-persistent buffers.
2024-12-24 14:32:11 +00:00
Ryan Dick
0a8fc74ae9
Add CachedModelWithPartialLoad to manage partially-loaded models using the new autocast modules.
2024-12-24 14:32:11 +00:00
Ryan Dick
dc54e8763b
Add CustomInvokeLinearNF4 to enable CPU -> GPU streaming for InvokeLinearNF4 layers.
2024-12-24 14:32:11 +00:00
Ryan Dick
1b56020876
Add CustomInvokeLinear8bitLt layer for device streaming with InvokeLinear8bitLt layers.
2024-12-24 14:32:11 +00:00
Ryan Dick
3f990393a1
Simplify the state management in InvokeLinear8bitLt and add unit tests. This is in preparation for wrapping it to support streaming of weights from cpu to gpu.
2024-12-24 14:32:11 +00:00
Ryan Dick
fe0ef2c27c
Add torch module autocast utilities.
2024-12-24 14:32:11 +00:00
Ryan Dick
65fcbf5f60
Bump bitsandbytes. The new verson contains improvements to state_dict loading/saving for LLM.int8 and promises improved speed on some HW.
2024-12-24 14:32:11 +00:00
Ryan Dick
55b13c1da3
(minor) Add TODO comment regarding the location of get_model_cache_key().
2024-12-24 14:23:19 +00:00
Ryan Dick
7dc3e0fdbe
Get rid of ModelLocker. It was an unnecessary layer of indirection.
2024-12-24 14:23:18 +00:00
Ryan Dick
a39bcf7e85
Move lock(...) and unlock(...) logic from ModelLocker to the ModelCache and make a bunch of ModelCache properties/methods private.
2024-12-24 14:23:18 +00:00
Ryan Dick
a7c72992a6
Pull get_model_cache_key(...) out of ModelCache. The ModelCache should not be concerned with implementation details like the submodel_type.
2024-12-24 14:23:18 +00:00
Ryan Dick
d30a9ced38
Rename model_cache_default.py -> model_cache.py.
2024-12-24 14:23:18 +00:00
Ryan Dick
e0bfa6157b
Remove ModelCacheBase.
2024-12-24 14:23:18 +00:00
Ryan Dick
83ea6420e2
Move CacheStats to its own file.
2024-12-24 14:23:18 +00:00
Ryan Dick
ce11a1952e
Move CacheRecord out to its own file.
2024-12-24 14:23:18 +00:00
Ryan Dick
e48dee4c4a
Rip out ModelLockerBase.
2024-12-24 14:23:18 +00:00
psychedelicious
948ecf9333
chore: bump version to v5.5.0
2024-12-20 16:17:23 +11:00
psychedelicious
1038f7bcab
Update invokeai_version.py
2024-12-20 10:17:09 +11:00
Riccardo Giovanetti
c7d9e2d62a
translationBot(ui): update translation (Italian)
...
Currently translated at 99.3% (1635 of 1645 strings)
translationBot(ui): update translation (Italian)
Currently translated at 99.3% (1634 of 1645 strings)
Co-authored-by: Riccardo Giovanetti <riccardo.giovanetti@gmail.com>
Translate-URL: https://hosted.weblate.org/projects/invokeai/web-ui/it/
Translation: InvokeAI/Web UI
2024-12-20 10:07:15 +11:00
Riku
11c3a2e15d
translationBot(ui): update translation (German)
...
Currently translated at 70.8% (1165 of 1645 strings)
Co-authored-by: Riku <riku.block@gmail.com>
Translate-URL: https://hosted.weblate.org/projects/invokeai/web-ui/de/
Translation: InvokeAI/Web UI
2024-12-20 10:07:15 +11:00
psychedelicious
9e3ca383ec
fix(ui): add missing model config to AnyModelConfig union type
2024-12-20 09:45:04 +11:00
Riku
bda83c2634
chore(ui): update typegen schema
2024-12-20 09:45:04 +11:00
Riku
525cb38c71
fix(app): fixed InputField default values
2024-12-20 09:30:56 +11:00
psychedelicious
a9a6720bad
feat(app): change queue item execution log from debug to info
...
This provides useful context for subsequent logs during queue item execution.
2024-12-20 09:19:04 +11:00
