Whilst the extracted faces are not required for convert (only the alignments file is), it is useful to have them so that we can clean up our dataset for the conversion process. There is technically a 3rd purpose which is when you are extracting for converting and want some of these faces for training as well. When generating an alignments file for converting, then the extracted faces are not actually required, but they are useful for cleaning the alignments file (more on this later). The alignment file contains information about where each face is in each frame so that the conversion process knows where to swap faces for any given frame.
Converting: To generate an alignments file and mask for converting your final frames. These faces will also contain the alignment information and masks which are required for training the model. Training: To generate a set of faces for training. Their pros and cons are detailed in the tooltips (for the GUI) or the help text (for the cli), so I won't be covering specifics. 
There are several plugins for each of these.
Mask Generation - Identify the parts of an aligned face that contains face and block out those areas that contain background/obstructions. 
It then attempts to use this information to align the face. This takes the candidates from the detector, and tries to work out where the key features (eyes, nose etc.) are on the potential face.
Alignment - Finding the "landmarks" (explained later) within a face and orienting the faces consistently. The detector scans an image and selects areas of the image that it thinks are faces. Detection - The process of finding faces within a frame. Why Extract?Īt the highest level, extraction consists of three phases: detection, alignment and mask generation: I will be using the GUI for this guide, but the premise is exactly the same for the cli (all the options present in the GUI are availablle in the cli). I'm not saying that this is the best workflow, or what you should do, but it works for me, and will hopefully give you a good jump off point for creating your own workflow. In this post I will detail a workflow for Extraction. It's how we learn, but it can sometimes help to have a bit of an understanding of the processes involved before diving in. Extracting a training set from an alignments fileĪ lot of people get overwhelmed when they start out Faceswapping, and many mistakes are made. NB: This guide was correct at time of writing, but things do change.
0 kommentar(er)
