AMV Post Production - Visual QualityYou have exported your video from Premiere (or some other editing package). It's been made progressive either by inverse telecine or by deinterlacing. Does it still look it's best? If you think it does, I bet it could still benefit from *some* filtering. Even some of the cleanest digital transfers can have problems. Almost every video could do with a little cleaning. However, if you are using a good DVD source you can probably gain much of the quality you need by using the new Post Processing options in mpeg2dec.dll when rendering (either when ripping clips or when exporting) as described in the footage preparation guides. These will clean up most of the common problems with mpeg2 compression by detecting them when decoding. It's really good :) You may still need filtering, however, and if you do then there are a few types of filters that you should be concerned with:
The programs that are designed to accommodate these filters are Virtualdub and Avisynth. In Virtualdub you can add filters, preview the settings and see a clear before and after image. However they do have a couple of problems:
Luckily for us, (practically) all virtualdub filters can be used in Avisynth! So, in order to keep everything in the same place, all of the filtering work in these guides will be done in Avisynth. The good thing about this is that you don't have to go through multiple filtered encodes to use filters from both programs and you can keep track of any colourspace changes. To make this even more easy, if you install the AdvancedAVS pack, it will copy a file called vdub_filters.avs which will define an avisynth command for practically every virtualdub filter known to man and you can use this command in your avisynth scripts. The virtualdub filters are stored in AVSFilters/vdubfilters/ so if you want to add more, copy them to that directory and there will probably be a function already defined for that filter, just read the vdub_filters.avs file in the AVSFilters directory to see what the function is for the virtualdub filter you want to use. OK, so all virtualdub filters in this guide will actually be used in avisynth. "But wait," I hear you cry "doesn't that mean reloading a script every time you want to see what a filter has done?!" Well, that used to be true until the invention of..... *drum roll* VirtualdubAVS This program is a godsend to anyone doing filtering work with avisynth. If you download the Advanced Avisynth pack and install it (you should have the AMVApp installed first) it includes a copy of VirtualdubAVS. For it to operate best, please locate your original Virtualdub folder when prompted by the installer. Otherwise it will be installed in the AMVapp folder (there is a direct link to wherever you installed it in Start Menu>Programs>AMVapp
With this program you can load an .avs file into it just like you would in virtualdub (using 'Open video file...') but then, by pressing Ctrl+E, a text editor box will appear so you can edit the script. What's even better is that once you have done an edit, you can press F5 and it will automatically save the script, refresh it in the window and go back to the exact frame you were looking at so you can see the changes! *swoon* This program also has a feature where you can load an existing media file (such as an avi or an mpeg) but load it via avisynth. You can select a template for reading the file and it will create an .avs file called theoriginalfilename--avisynth.avs - this can speed up your use of avisynth once you get used to it. There are templates for importing AVI (AVIsource), MPEG (DirectShowSource) and DVD2AVI file (with mpeg2dec). You can make more templates - they are stored in the templates subfolder of wherever you stored VirtualdubAVS. OK, the best way to get used to all of this stuff is to actually use it, so how's about we do some smoothing?
Spatial smoothing is a technique that takes a video source and attempts to get rid of various kinds of video noise in each individual still image. Each smooter has it's own little technique and pros and cons. The main two filters that you should be aware of for simple frame by frame noise removal are 2D Cleaner and SmartSmootherHiQ (which is a beefed up version of Smart Smoother). Both are very useful in their own right. Let's give one a go and you can see both how avisynth filtering works and how VirtualdubAVS makes it easy. Either make an avisynth script that loads an avi file of your choice or use the cunning 'load file via avisynth' option in VirtualdubAVS. If you've written your avisynth file (beginning with AVIsource or similar) load it into VirtualdubAVS with Load AVI File or just drag and drop. Press Ctrl-E. Here a text editing window will appear containing the script that is being viewed in VirtualdubAVS. In order to operate a filter you have to add a command to the end of this script (note: if you are not using the AMVapp you may have to include a loadplugin command at the top of yours script first). Note well - if there is a crash of any kind when using avisynth scripts, you may have to close VirtualdubAVS and reopen it for the crash to clear. Add the following lines to the end of your script:
OK, now I'll quickly explain what this does: 1) ConvertToYUY2() makes sure the video is in the right format for
an avisynth filter. If the source is already in the format it passes
through unchanged so there's no harm adding it except it will marginally
slow things down. The empty brackets are intentional. Now, change the number 10 to 100 and press F5. It should look really blurry :) Ideally what you want to achieve with these settings is to do a light blur on the source that will reduce noise without smearing the details too much. I find that small radius values (2x and 2y) are good with a strength of around 10 to 15. If your noise is large, then the blurring radius should be increased (slower). If you need to remove more noise then you need to increase the strength but be warned that for very noisy sources all you will end up doing is making the footage look like a watercolour painting. The other filter mentioned earlier is SmartSmootherHiQ which can be loaded with a command like this:
The words above would need to be replaced with numbers. There is an html file in the AdvancedAVS Docs folder (accessible from your start menu under AMVapp/AdvancedAVSDocs/) and this will explain the relevance of these functions and what good values are. Personally I prefer 2D cleaner for anime but SmoothHiQ can be good on sources with a lot of fine detail. You should experiment with the two and see which you prefer. Clever Spatial Smoothing There is one more filter that deserves particular merit, however. This filter is designed specifically with anime in mind. Unfortunately, because it is in early beta development it can only be used in RGB mode (which means that you should do it right after exporting from Premiere in RGB mode, before converting to YUY2 for other avisynth filters and on progressive sources). Also, it's not as fast as it could be because it is still experimental. In any case, it's worth mentioning because you may find a use for it. This filter is called msmooth and does three things. 1) It looks at the frame and finds the sharp edges 2) It creates a mask which will cover up these details on the image 3) it then smooths the areas that are not masked. This is great for digital noise such as macroblocking and mosquito noise. To use msmooth the command is as follows:
What you should do is start off by having mask=true. This is will show you which parts are considered "detail". You should then change the threshold and refresh with F5 until only the actual details of the frame are highlighted (and the noise is not). Once you have the right edge threshold, make mask=false and increase the smoothing strength until you are satisfied that all of the noise has gone. Of course, this filter is not perfect and it is sometimes difficult to find a balance with the edge threshold. Again, you should experiment with this filter but only use it if other filtering methods are not helping.
This is an excellent method of noise reduction but many people disregard it for the more logical spatial smoothers. Temporal Smoothing works by detecting if a pixel is changing because it needs to change or because there is some noise. If noise is detected then the pixels are either averaged out in time or the first pixel is copied to the subsequent frames. It has various ways of detecting whether it's noise or not, of course, but the result is that it can drastically reduce noise and even more drastically make the video more compressible as it is these subtle changes that we end up compressing. Possibly the best of the temporal smoothers is actually a virtualdub filter called Temporal Cleaner. As it's a vdub filter, it works in RGB so should be used on RGB files exported from premiere and before any YUY2 operations. The good thing is, this filter works on interlaced footage too, because it isn't smoothing spatially, so you can have it at the top of your script if you are need to remove interlacing using a YUY2 command. Of course, for this you should have your exported source in RGB mode but that's easy enough to do. To use Temporal Cleaner (as included in the AdvancedAVS pack) add this after the line that imports your file:
If you want to know what all these options mean, then by all means read the documentation (in Start Menu/Programs/AMVapp/AdvancedAVS Docs) but in all honesty the defaults are really good for most things. This filter can be quite slow at times and as a result, ErMaC recommends doing temporal cleaning on your exported file, encoding a new lossless copy first and then doing all your other filtering on this cleaned version, just because it saves time if you are encoding to lots of different formats in 2-pass mode. Smoothing in Time and Space Yep you guessed it, these filters do a bit of both. However, they aren't generally and all-purpose filtering solution designed to do everything and be better than the individual filters. They are, on the other hand, designed to filter your video in such a way as to reduce noise and be used beneficially on pretty much any source if you are compressing it. The two filters of significance in this category are Convolution3D and the STMedian Filter. Personally I find STMedian to be too powerful and not as effective as Convolution3D, so I'm not going to talk about it in detail. Convolution3D, on the other hand, I've found to be quite miraculous. Try this:
Now, you may well notice... um... not a lot. The instant visual effect of convolution3d on a frame is there, but it's not nearly as noticeable as a spatial smoother. However, if you compressed your video at constant maximum bitrate once with convolution3d and once without you'll notice that the convolution3d version is smaller! Hooray! This is good because it means that your compressor finds it much easier to compress a file that has been filtered with convolution3d, which means free quality. Also, you can make the filter stronger, which is good for noisier sources and very good for analogue sources - read the docs for more info. In short, convolution3d is a good thing and is welcome in every one of my encodes. If you want to play with the settings, the author has made some presets you may want to use. The setting above is a sort of Anime medium quality but instead you could use...
But really, the settings stated above will work well for most things - I'd merely try the settings for Low and Bad quality if you need to. Smoothers for analogue There are some smoothers that are designed for specific types of noise present in analogue sources. Convolution3D can help with this to a certain extent with aggressive values but here are some other filters which are useful for analogue sources, especially TV captures and VHS. The most notably useful one is called Chroma Noise Reducer 2 and is excellent on analogue source. It's an avisynth filter that is very simple to use - it operates in YUY2 colourspace and you run the filter like so:
Easy.
In terms of compression, sharpening is not usually a good thing :) It will make your file much harder to compress. However, there are situations where sharpening can be useful for improving picture quality. There are 2 sharpeners you need to know about for anime and they both give very different results. The first is Warpsharp which doesn't work like an ordinary sharpener as it sharpens through warping rather than convolution. To see what I mean, try it :) There are warpsharp versions for RGB mode and YUV mode included in the advanced avs pack - note well that the YUV version is very buggy and will have to be loaded manually:
or RGB
The benefit of warpsharp is in doing things like tidying up rough edges and eliminating dot crawl. It's a useful filter and probably best used *after* resizing but it really only has particular uses. I wouldn't use it on everything, only if there are edge problems in your video that could do with looking better. The other sharpening filter I'm going to talk about is msharpen - which is the brother of msmooth. Like msmooth, it does edge detection on the source and makes a mask but this time it masks out everything that isnt an edge and sharpens just the edges. This can lead to good results. It is very important, however, to correctly set up the edge detection.
Setup your threshold with mask=true and then put mask=false and set your strength. Small strength values are what you'll want - don't go OTT with this filter. It should have some uses for blurry sources.
Well, this one you are on your own somewhat because I can't advise on what settings to use because different sources require different solutions. What I can do is tell you the filters on offer and what they do. Documentation for these filters are included with the AdvancedAVS pack.
There are some filters that can be very useful to help quality when compressing your video that do not fall into the above categories but are still worthy of note. The first, and most important in terms of compression is a filter called Dup (included in the pack). Dup is another clever filter designed specifically for cell animation. See, cell animation isn't really 24 frames a second like movies - it's often only about 12fps. However, due to the nature of video, two originally identical frames are not always identical because of noise. If there is one pixel different from one frame to the next then the compressor will have to compress the macroblock containing that pixel. This happens a lot more than one would like. So, what Dup does is it detects to see if the last frame was originally the same as the current one (bar some slight noise). If it is the same frame, instead of using the one with slightly different noise it makes and exact duplicate of the first frame. This means that not a single pixel changes between frames and any inter-frame codec doesn't have to compress any of the frame. Add this up over the length of a whole amv and this can be a huge amount of bits saved. Sounds great, right? Well it is, when it works. The only problem is that sometimes the detection isn't sensitive enough. Computers cant tell things apart in quite the same way we do so two frames which obviously aren't supposed to be the same to us may fall within the margins of similarity to the computer. A scene where it's raining, the rain may be ignored as noise and frames are copied leaving the scene looking jerky, a very slow zoom in could also be too similar from one frame to the next and again the copying will become jerky. It's a pain. In short, there's no harm in trying if you've got the time. I tried this first on a usually 15mb compression and it came out 12mb with Dup - that's a huge saving with no visual loss in detail, but yes it doesn't work on all kinds of footage so you need to play around. Exotic Filters - Specific I think by now you probably have a fair grasp on how to use filters of various types so this is just a list of common visual problems and filters that are designed to help: Rainbow Artifacts/Moire
Dot Crawl
Chroma offset
If you need to find a specific filter for your needs, try searching this list for avisynth filters and this website for virtualdub filters.
|