i2p/Zerin-2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

initial commit

Browse Source
This commit is contained in:
i2p
2026-08-27 11:03:10 -06:00
commit d164820ea9
282 changed files with 90944 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
lib/.DS_Store
Vendored
BIN
View File
Binary file not shown.
lib/libjpeg-turbo/.gitattributes
Executable
+5
View File
@@ -0,0 +1,5 @@
/.gitattributes export-ignore
/.github export-ignore
*.ppm binary
*.pgm binary
/ChangeLog.md conflict-marker-size=8
lib/libjpeg-turbo/BUILDING.md
Executable
+703
View File
@@ -0,0 +1,703 @@
Building libjpeg-turbo
======================
Build Requirements
------------------
### All Systems
- [CMake](https://cmake.org) v2.8.12 or later
- [NASM](https://nasm.us) or [Yasm](https://yasm.tortall.net)
(if building x86 or x86-64 SIMD extensions)
* If using NASM, 2.13 or later is required.
* If using Yasm, 1.2.0 or later is required.
* NASM 2.15 or later is required if building libjpeg-turbo with Intel
Control-flow Enforcement Technology (CET) support.
* If building on macOS, NASM or Yasm can be obtained from
[MacPorts](https://macports.org) or [Homebrew](https://brew.sh).
- NOTE: Currently, if it is desirable to hide the SIMD function symbols in
Mac executables or shared libraries that statically link with
libjpeg-turbo, then NASM 2.14 or later or Yasm must be used when
building libjpeg-turbo.
* If NASM or Yasm is not in your `PATH`, then you can specify the full path
to the assembler by using either the `CMAKE_ASM_NASM_COMPILER` CMake
variable or the `ASM_NASM` environment variable. On Windows, use forward
slashes rather than backslashes in the path (for example,
**c:/nasm/nasm.exe**).
* NASM and Yasm are located in the CRB (Code Ready Builder) or PowerTools
repository on Red Hat Enterprise Linux 8+ and derivatives, which is not
enabled by default.
- If building the TurboJPEG Java wrapper, JDK or OpenJDK 1.5 or later is
required.
* Most modern Linux distributions, as well as Solaris 10 and later, include
JDK or OpenJDK. For other systems, pre-built JDK binaries can be obtained
from [Oracle](https://oracle.com/java/technologies/downloads) or
[Adoptium](https://adoptium.net/temurin/releases).
* If using JDK 11 or later, CMake 3.10.x or later must also be used.
### Un*x Platforms (including Mac and Cygwin)
- GCC v4.1 (or later) or Clang recommended for best performance
### Windows
- Microsoft Visual C++ 2005 or later
If you don't already have Visual C++, then the easiest way to get it is by
installing
[Visual Studio Community Edition](https://visualstudio.microsoft.com),
which includes everything necessary to build libjpeg-turbo.
* You can also download and install the standalone Windows SDK (for Windows 7
or later), which includes command-line versions of the 32-bit and 64-bit
Visual C++ compilers.
* If you intend to build libjpeg-turbo from the command line, then add the
appropriate compiler and SDK directories to the `INCLUDE`, `LIB`, and
`PATH` environment variables. This is generally accomplished by
executing `vcvars32.bat` or `vcvars64.bat`, which are located in the same
directory as the compiler.
* If built with Visual C++ 2015 or later, the libjpeg-turbo static libraries
cannot be used with earlier versions of Visual C++, and vice versa.
* The libjpeg API DLL (**jpeg{version}.dll**) will depend on the C run-time
DLLs corresponding to the version of Visual C++ that was used to build it.
... OR ...
- MinGW
[MSYS2](https://msys2.org) or [tdm-gcc](https://jmeubank.github.io/tdm-gcc)
recommended if building on a Windows machine. Both distributions install a
Start Menu link that can be used to launch a command prompt with the
appropriate compiler paths automatically set.
Sub-Project Builds
------------------
The libjpeg-turbo build system does not support being included as a sub-project
using the CMake `add_subdirectory()` function. Use the CMake
`ExternalProject_Add()` function instead.
Out-of-Tree Builds
------------------
Binary objects, libraries, and executables are generated in the directory from
which CMake is executed (the "binary directory"), and this directory need not
necessarily be the same as the libjpeg-turbo source directory. You can create
multiple independent binary directories, in which different versions of
libjpeg-turbo can be built from the same source tree using different compilers
or settings. In the sections below, *{build_directory}* refers to the binary
directory, whereas *{source_directory}* refers to the libjpeg-turbo source
directory. For in-tree builds, these directories are the same.
Ninja
-----
If using Ninja, then replace `make` or `nmake` with `ninja`, and replace the
CMake generator (specified with the `-G` option) with `Ninja`, in all of the
procedures and recipes below.
Build Procedure
---------------
NOTE: The build procedures below assume that CMake is invoked from the command
line, but all of these procedures can be adapted to the CMake GUI as
well.
### Un*x
The following procedure will build libjpeg-turbo on Unix and Unix-like systems.
(On Solaris, this generates a 32-bit build. See "Build Recipes" below for
64-bit build instructions.)
cd {build_directory}
cmake -G"Unix Makefiles" [additional CMake flags] {source_directory}
make
This will generate the following files under *{build_directory}*:
**libjpeg.a**<br>
Static link library for the libjpeg API
**libjpeg.so.{version}** (Linux, Unix)<br>
**libjpeg.{version}.dylib** (Mac)<br>
**cygjpeg-{version}.dll** (Cygwin)<br>
Shared library for the libjpeg API
By default, *{version}* is 62.2.0, 7.2.0, or 8.1.2, depending on whether
libjpeg v6b (default), v7, or v8 emulation is enabled. If using Cygwin,
*{version}* is 62, 7, or 8.
**libjpeg.so** (Linux, Unix)<br>
**libjpeg.dylib** (Mac)<br>
Development symlink for the libjpeg API
**libjpeg.dll.a** (Cygwin)<br>
Import library for the libjpeg API
**libturbojpeg.a**<br>
Static link library for the TurboJPEG API
**libturbojpeg.so.0.2.0** (Linux, Unix)<br>
**libturbojpeg.0.2.0.dylib** (Mac)<br>
**cygturbojpeg-0.dll** (Cygwin)<br>
Shared library for the TurboJPEG API
**libturbojpeg.so** (Linux, Unix)<br>
**libturbojpeg.dylib** (Mac)<br>
Development symlink for the TurboJPEG API
**libturbojpeg.dll.a** (Cygwin)<br>
Import library for the TurboJPEG API
### Visual C++ (Command Line)
cd {build_directory}
cmake -G"NMake Makefiles" -DCMAKE_BUILD_TYPE=Release [additional CMake flags] {source_directory}
nmake
This will build either a 32-bit or a 64-bit version of libjpeg-turbo, depending
on which version of **cl.exe** is in the `PATH`.
The following files will be generated under *{build_directory}*:
**jpeg-static.lib**<br>
Static link library for the libjpeg API
**jpeg{version}.dll**<br>
DLL for the libjpeg API
**jpeg.lib**<br>
Import library for the libjpeg API
**turbojpeg-static.lib**<br>
Static link library for the TurboJPEG API
**turbojpeg.dll**<br>
DLL for the TurboJPEG API
**turbojpeg.lib**<br>
Import library for the TurboJPEG API
*{version}* is 62, 7, or 8, depending on whether libjpeg v6b (default), v7, or
v8 emulation is enabled.
### Visual C++ (IDE)
Choose the appropriate CMake generator option for your version of Visual Studio
(run `cmake` with no arguments for a list of available generators.) For
instance:
cd {build_directory}
cmake -G"Visual Studio 10" [additional CMake flags] {source_directory}
NOTE: Add "Win64" to the generator name (for example, "Visual Studio 10 Win64")
to build a 64-bit version of libjpeg-turbo. A separate build directory must be
used for 32-bit and 64-bit builds.
You can then open **ALL_BUILD.vcproj** in Visual Studio and build one of the
configurations in that project ("Debug", "Release", etc.) to generate a full
build of libjpeg-turbo.
This will generate the following files under *{build_directory}*:
**{configuration}/jpeg-static.lib**<br>
Static link library for the libjpeg API
**{configuration}/jpeg{version}.dll**<br>
DLL for the libjpeg API
**{configuration}/jpeg.lib**<br>
Import library for the libjpeg API
**{configuration}/turbojpeg-static.lib**<br>
Static link library for the TurboJPEG API
**{configuration}/turbojpeg.dll**<br>
DLL for the TurboJPEG API
**{configuration}/turbojpeg.lib**<br>
Import library for the TurboJPEG API
*{configuration}* is Debug, Release, RelWithDebInfo, or MinSizeRel, depending
on the configuration you built in the IDE, and *{version}* is 62, 7, or 8,
depending on whether libjpeg v6b (default), v7, or v8 emulation is enabled.
### MinGW
NOTE: This assumes that you are building on a Windows machine using the MSYS
environment. If you are cross-compiling on a Un*x platform (including Mac and
Cygwin), then see "Build Recipes" below.
cd {build_directory}
cmake -G"MSYS Makefiles" [additional CMake flags] {source_directory}
make
This will generate the following files under *{build_directory}*:
**libjpeg.a**<br>
Static link library for the libjpeg API
**libjpeg-{version}.dll**<br>
DLL for the libjpeg API
**libjpeg.dll.a**<br>
Import library for the libjpeg API
**libturbojpeg.a**<br>
Static link library for the TurboJPEG API
**libturbojpeg.dll**<br>
DLL for the TurboJPEG API
**libturbojpeg.dll.a**<br>
Import library for the TurboJPEG API
*{version}* is 62, 7, or 8, depending on whether libjpeg v6b (default), v7, or
v8 emulation is enabled.
### Debug Build
Add `-DCMAKE_BUILD_TYPE=Debug` to the CMake command line. Or, if building
with NMake, remove `-DCMAKE_BUILD_TYPE=Release` (Debug builds are the default
with NMake.)
### libjpeg v7 or v8 API/ABI Emulation
Add `-DWITH_JPEG7=1` to the CMake command line to build a version of
libjpeg-turbo that is API/ABI-compatible with libjpeg v7. Add `-DWITH_JPEG8=1`
to the CMake command line to build a version of libjpeg-turbo that is
API/ABI-compatible with libjpeg v8. See [README.md](README.md) for more
information about libjpeg v7 and v8 emulation.
### Arithmetic Coding Support
Since the patent on arithmetic coding has expired, this functionality has been
included in this release of libjpeg-turbo. libjpeg-turbo's implementation is
based on the implementation in libjpeg v8, but it works when emulating libjpeg
v7 or v6b as well. The default is to enable both arithmetic encoding and
decoding, but those who have philosophical objections to arithmetic coding can
add `-DWITH_ARITH_ENC=0` or `-DWITH_ARITH_DEC=0` to the CMake command line to
disable encoding or decoding (respectively.)
### TurboJPEG Java Wrapper
Add `-DWITH_JAVA=1` to the CMake command line to incorporate an optional Java
Native Interface (JNI) wrapper into the TurboJPEG shared library and build the
Java front-end classes to support it. This allows the TurboJPEG shared library
to be used directly from Java applications. See
[java/README.md](java/README.md) for more details.
If Java is not in your `PATH`, or if you wish to use an alternate JDK to
build/test libjpeg-turbo, then (prior to running CMake) set the `JAVA_HOME`
environment variable to the location of the JDK that you wish to use. The
`Java_JAVAC_EXECUTABLE`, `Java_JAVA_EXECUTABLE`, and `Java_JAR_EXECUTABLE`
CMake variables can also be used to specify alternate commands or locations for
javac, jar, and java (respectively.) You can also set the
`CMAKE_JAVA_COMPILE_FLAGS` CMake variable or the `JAVAFLAGS` environment
variable to specify arguments that should be passed to the Java compiler when
building the TurboJPEG classes, and the `JAVAARGS` CMake variable to specify
arguments that should be passed to the JRE when running the TurboJPEG Java unit
tests.
Build Recipes
-------------
### 32-bit Build on 64-bit Linux/Unix
Use export/setenv to set the following environment variables before running
CMake:
CFLAGS=-m32
LDFLAGS=-m32
### 64-bit Build on Solaris
Use export/setenv to set the following environment variables before running
CMake:
CFLAGS=-m64
LDFLAGS=-m64
### Other Compilers
On Un*x systems, prior to running CMake, you can set the `CC` environment
variable to the command used to invoke the C compiler.
### 32-bit MinGW Build on Un*x (including Mac and Cygwin)
Create a file called **toolchain.cmake** under *{build_directory}*, with the
following contents:
set(CMAKE_SYSTEM_NAME Windows)
set(CMAKE_SYSTEM_PROCESSOR X86)
set(CMAKE_C_COMPILER {mingw_binary_path}/i686-w64-mingw32-gcc)
set(CMAKE_RC_COMPILER {mingw_binary_path}/i686-w64-mingw32-windres)
*{mingw\_binary\_path}* is the directory under which the MinGW binaries are
located (usually **/usr/bin**.) Next, execute the following commands:
cd {build_directory}
cmake -G"Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE=toolchain.cmake \
-DCMAKE_INSTALL_PREFIX={install_path} \
[additional CMake flags] {source_directory}
make
*{install\_path}* is the path under which the libjpeg-turbo binaries should be
installed.
### 64-bit MinGW Build on Un*x (including Mac and Cygwin)
Create a file called **toolchain.cmake** under *{build_directory}*, with the
following contents:
set(CMAKE_SYSTEM_NAME Windows)
set(CMAKE_SYSTEM_PROCESSOR AMD64)
set(CMAKE_C_COMPILER {mingw_binary_path}/x86_64-w64-mingw32-gcc)
set(CMAKE_RC_COMPILER {mingw_binary_path}/x86_64-w64-mingw32-windres)
*{mingw\_binary\_path}* is the directory under which the MinGW binaries are
located (usually **/usr/bin**.) Next, execute the following commands:
cd {build_directory}
cmake -G"Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE=toolchain.cmake \
-DCMAKE_INSTALL_PREFIX={install_path} \
[additional CMake flags] {source_directory}
make
*{install\_path}* is the path under which the libjpeg-turbo binaries should be
installed.
Building libjpeg-turbo for iOS
------------------------------
iOS platforms, such as the iPhone and iPad, use Arm processors, and all
currently supported models include Neon instructions. Thus, they can take
advantage of libjpeg-turbo's SIMD extensions to significantly accelerate JPEG
compression/decompression. This section describes how to build libjpeg-turbo
for these platforms.
### Armv8 (64-bit)
**Xcode 5 or later required, Xcode 6.3.x or later recommended**
The following script demonstrates how to build libjpeg-turbo to run on the
iPhone 5S/iPad Mini 2/iPad Air and newer.
IOS_PLATFORMDIR=/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform
IOS_SYSROOT=($IOS_PLATFORMDIR/Developer/SDKs/iPhoneOS*.sdk)
export CFLAGS="-Wall -miphoneos-version-min=8.0 -funwind-tables"
cd {build_directory}
cmake -G"Unix Makefiles" \
-DCMAKE_C_COMPILER=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang \
-DCMAKE_OSX_ARCHITECTURES=arm64 \
-DCMAKE_OSX_SYSROOT=${IOS_SYSROOT[0]} \
[additional CMake flags] {source_directory}
make
Replace `iPhoneOS` with `iPhoneSimulator` and `-miphoneos-version-min` with
`-miphonesimulator-version-min` to build libjpeg-turbo for the iOS simulator on
Macs with Apple silicon CPUs.
Building libjpeg-turbo for Android
----------------------------------
Building libjpeg-turbo for Android platforms requires v13b or later of the
[Android NDK](https://developer.android.com/ndk).
### Armv7 (32-bit)
**NDK r19 or later with Clang recommended**
The following is a general recipe script that can be modified for your specific
needs.
# Set these variables to suit your needs
NDK_PATH={full path to the NDK directory-- for example,
/opt/android/android-ndk-r16b}
TOOLCHAIN={"gcc" or "clang"-- "gcc" must be used with NDK r16b and earlier,
and "clang" must be used with NDK r17c and later}
ANDROID_VERSION={the minimum version of Android to support-- for example,
"16", "19", etc.}
cd {build_directory}
cmake -G"Unix Makefiles" \
-DANDROID_ABI=armeabi-v7a \
-DANDROID_ARM_MODE=arm \
-DANDROID_PLATFORM=android-${ANDROID_VERSION} \
-DANDROID_TOOLCHAIN=${TOOLCHAIN} \
-DCMAKE_ASM_FLAGS="--target=arm-linux-androideabi${ANDROID_VERSION}" \
-DCMAKE_TOOLCHAIN_FILE=${NDK_PATH}/build/cmake/android.toolchain.cmake \
[additional CMake flags] {source_directory}
make
### Armv8 (64-bit)
**Clang recommended**
The following is a general recipe script that can be modified for your specific
needs.
# Set these variables to suit your needs
NDK_PATH={full path to the NDK directory-- for example,
/opt/android/android-ndk-r16b}
TOOLCHAIN={"gcc" or "clang"-- "gcc" must be used with NDK r14b and earlier,
and "clang" must be used with NDK r17c and later}
ANDROID_VERSION={the minimum version of Android to support. "21" or later
is required for a 64-bit build.}
cd {build_directory}
cmake -G"Unix Makefiles" \
-DANDROID_ABI=arm64-v8a \
-DANDROID_ARM_MODE=arm \
-DANDROID_PLATFORM=android-${ANDROID_VERSION} \
-DANDROID_TOOLCHAIN=${TOOLCHAIN} \
-DCMAKE_ASM_FLAGS="--target=aarch64-linux-android${ANDROID_VERSION}" \
-DCMAKE_TOOLCHAIN_FILE=${NDK_PATH}/build/cmake/android.toolchain.cmake \
[additional CMake flags] {source_directory}
make
### x86 (32-bit)
The following is a general recipe script that can be modified for your specific
needs.
# Set these variables to suit your needs
NDK_PATH={full path to the NDK directory-- for example,
/opt/android/android-ndk-r16b}
TOOLCHAIN={"gcc" or "clang"-- "gcc" must be used with NDK r14b and earlier,
and "clang" must be used with NDK r17c and later}
ANDROID_VERSION={The minimum version of Android to support-- for example,
"16", "19", etc.}
cd {build_directory}
cmake -G"Unix Makefiles" \
-DANDROID_ABI=x86 \
-DANDROID_PLATFORM=android-${ANDROID_VERSION} \
-DANDROID_TOOLCHAIN=${TOOLCHAIN} \
-DCMAKE_TOOLCHAIN_FILE=${NDK_PATH}/build/cmake/android.toolchain.cmake \
[additional CMake flags] {source_directory}
make
### x86-64 (64-bit)
The following is a general recipe script that can be modified for your specific
needs.
# Set these variables to suit your needs
NDK_PATH={full path to the NDK directory-- for example,
/opt/android/android-ndk-r16b}
TOOLCHAIN={"gcc" or "clang"-- "gcc" must be used with NDK r14b and earlier,
and "clang" must be used with NDK r17c and later}
ANDROID_VERSION={the minimum version of Android to support. "21" or later
is required for a 64-bit build.}
cd {build_directory}
cmake -G"Unix Makefiles" \
-DANDROID_ABI=x86_64 \
-DANDROID_PLATFORM=android-${ANDROID_VERSION} \
-DANDROID_TOOLCHAIN=${TOOLCHAIN} \
-DCMAKE_TOOLCHAIN_FILE=${NDK_PATH}/build/cmake/android.toolchain.cmake \
[additional CMake flags] {source_directory}
make
Advanced CMake Options
----------------------
To list and configure other CMake options not specifically mentioned in this
guide, run
ccmake {source_directory}
or
cmake-gui {source_directory}
from the build directory after initially configuring the build. CCMake is a
text-based interactive version of CMake, and CMake-GUI is a GUI version. Both
will display all variables that are relevant to the libjpeg-turbo build, their
current values, and a help string describing what they do.
Installing libjpeg-turbo
========================
You can use the build system to install libjpeg-turbo (as opposed to creating
an installer package.) To do this, run `make install` or `nmake install`
(or build the "install" target in the Visual Studio IDE.) Running
`make uninstall` or `nmake uninstall` (or building the "uninstall" target in
the Visual Studio IDE) will uninstall libjpeg-turbo.
The `CMAKE_INSTALL_PREFIX` CMake variable can be modified in order to install
libjpeg-turbo into a directory of your choosing. If you don't specify
`CMAKE_INSTALL_PREFIX`, then the default is:
**c:\libjpeg-turbo**<br>
Visual Studio 32-bit build
**c:\libjpeg-turbo64**<br>
Visual Studio 64-bit build
**c:\libjpeg-turbo-gcc**<br>
MinGW 32-bit build
**c:\libjpeg-turbo-gcc64**<br>
MinGW 64-bit build
**/opt/libjpeg-turbo**<br>
Un*x (including Mac and Cygwin)
The default value of `CMAKE_INSTALL_PREFIX` causes the libjpeg-turbo files to
be installed with a directory structure resembling that of the official
libjpeg-turbo binary packages. Changing the value of `CMAKE_INSTALL_PREFIX`
(for instance, to **/usr/local**) causes the libjpeg-turbo files to be
installed with a directory structure that conforms to GNU standards.
The `CMAKE_INSTALL_BINDIR`, `CMAKE_INSTALL_DATAROOTDIR`,
`CMAKE_INSTALL_DOCDIR`, `CMAKE_INSTALL_INCLUDEDIR`, `CMAKE_INSTALL_JAVADIR`,
`CMAKE_INSTALL_LIBDIR`, and `CMAKE_INSTALL_MANDIR` CMake variables allow a
finer degree of control over where specific files in the libjpeg-turbo
distribution should be installed. These directory variables can either be
specified as absolute paths or as paths relative to `CMAKE_INSTALL_PREFIX` (for
instance, setting `CMAKE_INSTALL_DOCDIR` to **doc** would cause the
documentation to be installed in **${CMAKE\_INSTALL\_PREFIX}/doc**.) If a
directory variable contains the name of another directory variable in angle
brackets, then its final value will depend on the final value of that other
variable. For instance, the default value of `CMAKE_INSTALL_MANDIR` is
**\<CMAKE\_INSTALL\_DATAROOTDIR\>/man**.
Creating Distribution Packages
==============================
The following commands can be used to create various types of distribution
packages:
Linux
-----
make rpm
Create Red Hat-style binary RPM package. Requires RPM v4 or later.
make srpm
This runs `make dist` to create a pristine source tarball, then creates a
Red Hat-style source RPM package from the tarball. Requires RPM v4 or later.
make deb
Create Debian-style binary package. Requires dpkg.
Mac
---
make dmg
Create Mac package/disk image. This requires pkgbuild and productbuild, which
are installed by default on OS X/macOS 10.7 and later.
In order to create a Mac package/disk image that contains universal
x86-64/Arm binaries, set the following CMake variable:
* `SECONDARY_BUILD`: Directory containing a cross-compiled x86-64 or Armv8
(64-bit) iOS or macOS build of libjpeg-turbo to include in the universal
binaries
You should first use CMake to configure the cross-compiled x86-64 or Armv8
secondary build of libjpeg-turbo (see "Building libjpeg-turbo for iOS" above,
if applicable) in a build directory that matches the one specified in the
aforementioned CMake variable. Next, configure the primary (native) build of
libjpeg-turbo as an out-of-tree build, specifying the aforementioned CMake
variable, and build it. Once the primary build has been built, run `make dmg`
from the build directory. The packaging system will build the secondary build,
use lipo to combine it with the primary build into a single set of universal
binaries, then package the universal binaries.
Windows
-------
If using NMake:
cd {build_directory}
nmake installer
If using MinGW:
cd {build_directory}
make installer
If using the Visual Studio IDE, build the "installer" target.
The installer package (libjpeg-turbo-*{version}*[-gcc|-vc][64].exe) will be
located under *{build_directory}*. If building using the Visual Studio IDE,
then the installer package will be located in a subdirectory with the same name
as the configuration you built (such as *{build_directory}*\Debug\ or
*{build_directory}*\Release\).
Building a Windows installer requires the
[Nullsoft Install System](https://nsis.sourceforge.io). makensis.exe should
be in your `PATH`.
Regression testing
==================
The most common way to test libjpeg-turbo is by invoking `make test` (Un*x) or
`nmake test` (Windows command line) or by building the "RUN_TESTS" target
(Visual Studio IDE), once the build has completed. This runs a series of tests
to ensure that mathematical compatibility has been maintained between
libjpeg-turbo and libjpeg v6b. This also invokes the TurboJPEG unit tests,
which ensure that the colorspace extensions, YUV encoding, decompression
scaling, and other features of the TurboJPEG C and Java APIs are working
properly (and, by extension, that the equivalent features of the underlying
libjpeg API are also working.)
Invoking `make testclean` (Un*x) or `nmake testclean` (Windows command line) or
building the "testclean" target (Visual Studio IDE) will clean up the output
images generated by the tests.
On Un*x platforms, more extensive tests of the TurboJPEG C and Java wrappers
can be run by invoking `make tjtest`, `make tjtest12`, and `make tjtest16`.
These extended TurboJPEG tests essentially iterate through all of the available
features of the TurboJPEG APIs that are not covered by the TurboJPEG unit tests
(including the lossless transform options) and compare the images generated by
each feature to images generated using the equivalent feature in the libjpeg
API. The extended TurboJPEG tests are meant to test for regressions in the
TurboJPEG wrappers, not in the underlying libjpeg API library.
lib/libjpeg-turbo/CMakeLists.txt
Executable
+1946
View File
File diff suppressed because it is too large Load Diff
lib/libjpeg-turbo/ChangeLog.md
Executable
+2506
View File
File diff suppressed because it is too large Load Diff
lib/libjpeg-turbo/LICENSE.md
Executable
+135
View File
@@ -0,0 +1,135 @@
libjpeg-turbo Licenses
======================
libjpeg-turbo is covered by two compatible BSD-style open source licenses:
- The IJG (Independent JPEG Group) License, which is listed in
[README.ijg](README.ijg)
This license applies to the libjpeg API library and associated programs,
including any code inherited from libjpeg and any modifications to that
code. Note that the libjpeg-turbo SIMD source code bears the
[zlib License](https://opensource.org/licenses/Zlib), but in the context of
the overall libjpeg API library, the terms of the zlib License are subsumed
by the terms of the IJG License.
- The Modified (3-clause) BSD License, which is listed below
This license applies to the TurboJPEG API library and associated programs, as
well as the build system. Note that the TurboJPEG API library wraps the
libjpeg API library, so in the context of the overall TurboJPEG API library,
both the terms of the IJG License and the terms of the Modified (3-clause)
BSD License apply.
Complying with the libjpeg-turbo Licenses
=========================================
This section provides a roll-up of the libjpeg-turbo licensing terms, to the
best of our understanding. This is not a license in and of itself. It is
intended solely for clarification.
1. If you are distributing a modified version of the libjpeg-turbo source,
then:
1. You cannot alter or remove any existing copyright or license notices
from the source.
**Origin**
- Clause 1 of the IJG License
- Clause 1 of the Modified BSD License
- Clauses 1 and 3 of the zlib License
2. You must add your own copyright notice to the header of each source
file you modified, so others can tell that you modified that file. (If
there is not an existing copyright header in that file, then you can
simply add a notice stating that you modified the file.)
**Origin**
- Clause 1 of the IJG License
- Clause 2 of the zlib License
3. You must include the IJG README file, and you must not alter any of the
copyright or license text in that file.
**Origin**
- Clause 1 of the IJG License
2. If you are distributing only libjpeg-turbo binaries without the source, or
if you are distributing an application that statically links with
libjpeg-turbo, then:
1. Your product documentation must include a message stating:
This software is based in part on the work of the Independent JPEG
Group.
**Origin**
- Clause 2 of the IJG license
2. If your binary distribution includes or uses the TurboJPEG API, then
your product documentation must include the text of the Modified BSD
License (see below.)
**Origin**
- Clause 2 of the Modified BSD License
3. You cannot use the name of the IJG or The libjpeg-turbo Project or the
contributors thereof in advertising, publicity, etc.
**Origin**
- IJG License
- Clause 3 of the Modified BSD License
4. The IJG and The libjpeg-turbo Project do not warrant libjpeg-turbo to be
free of defects, nor do we accept any liability for undesirable
consequences resulting from your use of the software.
**Origin**
- IJG License
- Modified BSD License
- zlib License
The Modified (3-clause) BSD License
===================================
Copyright (C) 2009-2026 D. R. Commander. All Rights Reserved.<br>
Copyright (C) 2015 Viktor Szathmáry. All Rights Reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
- Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
- Neither the name of the libjpeg-turbo Project nor the names of its
contributors may be used to endorse or promote products derived from this
software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS",
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
Why Two Licenses?
=================
The zlib License could have been used instead of the Modified (3-clause) BSD
License, and since the IJG License effectively subsumes the distribution
conditions of the zlib License, this would have effectively placed
libjpeg-turbo binary distributions under the IJG License. However, the IJG
License specifically refers to the Independent JPEG Group and does not extend
attribution and endorsement protections to other entities. Thus, it was
desirable to choose a license that granted us the same protections for new code
that were granted to the IJG for code derived from their software.
lib/libjpeg-turbo/README.ijg
Executable
+260
View File
@@ -0,0 +1,260 @@
libjpeg-turbo note: This file has been modified by The libjpeg-turbo Project
to include only information relevant to libjpeg-turbo, to wordsmith certain
sections, and to remove impolitic language that existed in the libjpeg v8
README. It is included only for reference. Please see README.md for
information specific to libjpeg-turbo.
The Independent JPEG Group's JPEG software
==========================================
This distribution contains a release of the Independent JPEG Group's free JPEG
software. You are welcome to redistribute this software and to use it for any
purpose, subject to the conditions under LEGAL ISSUES, below.
This software is the work of Tom Lane, Guido Vollbeding, Philip Gladstone,
Bill Allombert, Jim Boucher, Lee Crocker, Bob Friesenhahn, Ben Jackson,
Julian Minguillon, Luis Ortiz, George Phillips, Davide Rossi, Ge' Weijers,
and other members of the Independent JPEG Group.
IJG is not affiliated with the ISO/IEC JTC1/SC29/WG1 standards committee
(also known as JPEG, together with ITU-T SG16).
DOCUMENTATION ROADMAP
=====================
This file contains the following sections:
OVERVIEW General description of JPEG and the IJG software.
LEGAL ISSUES Copyright, lack of warranty, terms of distribution.
REFERENCES Where to learn more about JPEG.
ARCHIVE LOCATIONS Where to find newer versions of this software.
FILE FORMAT WARS Software *not* to get.
TO DO Plans for future IJG releases.
Other documentation files in the distribution are:
User documentation:
doc/usage.txt Usage instructions for cjpeg, djpeg, jpegtran,
rdjpgcom, and wrjpgcom.
doc/*.1 Unix-style man pages for programs (same info as
usage.txt).
doc/wizard.txt Advanced usage instructions for JPEG wizards only.
doc/change.log Version-to-version change highlights.
Programmer and internal documentation:
doc/libjpeg.txt How to use the JPEG library in your own programs.
src/example.c Sample code for calling the JPEG library.
doc/structure.txt Overview of the JPEG library's internal structure.
doc/coderules.txt Coding style rules --- please read if you contribute
code.
Please read at least usage.txt. Some information can also be found in the JPEG
FAQ (Frequently Asked Questions) article. See ARCHIVE LOCATIONS below to find
out where to obtain the FAQ article.
If you want to understand how the JPEG code works, we suggest reading one or
more of the REFERENCES, then looking at the documentation files (in roughly
the order listed) before diving into the code.
OVERVIEW
========
This package contains C software to implement JPEG image encoding, decoding,
and transcoding. JPEG (pronounced "jay-peg") is a standardized compression
method for full-color and grayscale images. JPEG's strong suit is compressing
photographic images or other types of images that have smooth color and
brightness transitions between neighboring pixels. Images with sharp lines or
other abrupt features may not compress well with JPEG, and a higher JPEG
quality may have to be used to avoid visible compression artifacts with such
images.
JPEG is normally lossy, meaning that the output pixels are not necessarily
identical to the input pixels. However, on photographic content and other
"smooth" images, very good compression ratios can be obtained with no visible
compression artifacts, and extremely high compression ratios are possible if
you are willing to sacrifice image quality (by reducing the "quality" setting
in the compressor.)
This software implements JPEG baseline, extended-sequential, progressive, and
lossless compression processes. Provision is made for supporting all variants
of these processes, although some uncommon parameter settings aren't
implemented yet. We have made no provision for supporting the hierarchical
processes defined in the standard.
We provide a set of library routines for reading and writing JPEG image files,
plus two sample applications "cjpeg" and "djpeg", which use the library to
perform conversion between JPEG and some other popular image file formats.
The library is intended to be reused in other applications.
In order to support file conversion and viewing software, we have included
considerable functionality beyond the bare JPEG coding/decoding capability;
for example, the color quantization modules are not strictly part of JPEG
decoding, but they are essential for output to colormapped file formats. These
extra functions can be compiled out of the library if not required for a
particular application.
We have also included "jpegtran", a utility for lossless transcoding between
different JPEG processes, and "rdjpgcom" and "wrjpgcom", two simple
applications for inserting and extracting textual comments in JFIF files.
The emphasis in designing this software has been on achieving portability and
flexibility, while also making it fast enough to be useful. In particular,
the software is not intended to be read as a tutorial on JPEG. (See the
REFERENCES section for introductory material.) Rather, it is intended to
be reliable, portable, industrial-strength code. We do not claim to have
achieved that goal in every aspect of the software, but we strive for it.
We welcome the use of this software as a component of commercial products.
No royalty is required, but we do ask for an acknowledgement in product
documentation, as described under LEGAL ISSUES.
LEGAL ISSUES
============
In plain English:
1. We don't promise that this software works. (But if you find any bugs,
please let us know!)
2. You can use this software for whatever you want. You don't have to pay us.
3. You may not pretend that you wrote this software. If you use it in a
program, you must acknowledge somewhere in your documentation that
you've used the IJG code.
In legalese:
The authors make NO WARRANTY or representation, either express or implied,
with respect to this software, its quality, accuracy, merchantability, or
fitness for a particular purpose. This software is provided "AS IS", and you,
its user, assume the entire risk as to its quality and accuracy.
This software is copyright (C) 1991-2020, Thomas G. Lane, Guido Vollbeding.
All Rights Reserved except as specified below.
Permission is hereby granted to use, copy, modify, and distribute this
software (or portions thereof) for any purpose, without fee, subject to these
conditions:
(1) If any part of the source code for this software is distributed, then this
README file must be included, with this copyright and no-warranty notice
unaltered; and any additions, deletions, or changes to the original files
must be clearly indicated in accompanying documentation.
(2) If only executable code is distributed, then the accompanying
documentation must state that "this software is based in part on the work of
the Independent JPEG Group".
(3) Permission for use of this software is granted only if the user accepts
full responsibility for any undesirable consequences; the authors accept
NO LIABILITY for damages of any kind.
These conditions apply to any software derived from or based on the IJG code,
not just to the unmodified library. If you use our work, you ought to
acknowledge us.
Permission is NOT granted for the use of any IJG author's name or company name
in advertising or publicity relating to this software or products derived from
it. This software may be referred to only as "the Independent JPEG Group's
software".
We specifically permit and encourage the use of this software as the basis of
commercial products, provided that all warranty or liability claims are
assumed by the product vendor.
REFERENCES
==========
We recommend reading one or more of these references before trying to
understand the innards of the JPEG software.
The best short technical introduction to the JPEG compression algorithm is
Wallace, Gregory K. "The JPEG Still Picture Compression Standard",
Communications of the ACM, April 1991 (vol. 34 no. 4), pp. 30-44.
(Adjacent articles in that issue discuss MPEG motion picture compression,
applications of JPEG, and related topics.) If you don't have the CACM issue
handy, a PDF file containing a revised version of Wallace's article is
available at http://www.ijg.org/files/Wallace.JPEG.pdf. The file (actually
a preprint for an article that appeared in IEEE Trans. Consumer Electronics)
omits the sample images that appeared in CACM, but it includes corrections
and some added material. Note: the Wallace article is copyright ACM and IEEE,
and it may not be used for commercial purposes.
A somewhat less technical, more leisurely introduction to JPEG can be found in
"The Data Compression Book" by Mark Nelson and Jean-loup Gailly, published by
M&T Books (New York), 2nd ed. 1996, ISBN 1-55851-434-1. This book provides
good explanations and example C code for a multitude of compression methods
including JPEG. It is an excellent source if you are comfortable reading C
code but don't know much about data compression in general. The book's JPEG
sample code is far from industrial-strength, but when you are ready to look
at a full implementation, you've got one here...
The best currently available description of JPEG is the textbook "JPEG Still
Image Data Compression Standard" by William B. Pennebaker and Joan L.
Mitchell, published by Van Nostrand Reinhold, 1993, ISBN 0-442-01272-1.
Price US$59.95, 638 pp. The book includes the complete text of the ISO JPEG
standards (DIS 10918-1 and draft DIS 10918-2).
The original JPEG standard is divided into two parts, Part 1 being the actual
specification, while Part 2 covers compliance testing methods. Part 1 is
titled "Digital Compression and Coding of Continuous-tone Still Images,
Part 1: Requirements and guidelines" and has document numbers ISO/IEC IS
10918-1, ITU-T T.81. Part 2 is titled "Digital Compression and Coding of
Continuous-tone Still Images, Part 2: Compliance testing" and has document
numbers ISO/IEC IS 10918-2, ITU-T T.83.
The JPEG standard does not specify all details of an interchangeable file
format. For the omitted details, we follow the "JFIF" conventions, revision
1.02. JFIF version 1 has been adopted as ISO/IEC 10918-5 (05/2013) and
Recommendation ITU-T T.871 (05/2011): Information technology - Digital
compression and coding of continuous-tone still images: JPEG File Interchange
Format (JFIF). It is available as a free download in PDF file format from
https://www.iso.org/standard/54989.html and http://www.itu.int/rec/T-REC-T.871.
A PDF file of the older JFIF 1.02 specification is available at
http://www.w3.org/Graphics/JPEG/jfif3.pdf.
The TIFF 6.0 file format specification can be obtained from
http://mirrors.ctan.org/graphics/tiff/TIFF6.ps.gz. The JPEG incorporation
scheme found in the TIFF 6.0 spec of 3-June-92 has a number of serious
problems. IJG does not recommend use of the TIFF 6.0 design (TIFF Compression
tag 6). Instead, we recommend the JPEG design proposed by TIFF Technical Note
#2 (Compression tag 7). Copies of this Note can be obtained from
http://www.ijg.org/files/. It is expected that the next revision
of the TIFF spec will replace the 6.0 JPEG design with the Note's design.
Although IJG's own code does not support TIFF/JPEG, the free libtiff library
uses our library to implement TIFF/JPEG per the Note.
ARCHIVE LOCATIONS
=================
The "official" archive site for this software is www.ijg.org.
The most recent released version can always be found there in
directory "files".
The JPEG FAQ (Frequently Asked Questions) article is a source of some
general information about JPEG. It is available at
http://www.faqs.org/faqs/jpeg-faq.
FILE FORMAT COMPATIBILITY
=========================
This software implements ITU T.81 | ISO/IEC 10918 with some extensions from
ITU T.871 | ISO/IEC 10918-5 (JPEG File Interchange Format-- see REFERENCES).
Informally, the term "JPEG image" or "JPEG file" most often refers to JFIF or
a subset thereof, but there are other formats containing the name "JPEG" that
are incompatible with the original JPEG standard or with JFIF (for instance,
JPEG 2000 and JPEG XR). This software therefore does not support these
formats. Indeed, one of the original reasons for developing this free software
was to help force convergence on a common, interoperable format standard for
JPEG files.
JFIF is a minimal or "low end" representation. TIFF/JPEG (TIFF revision 6.0 as
modified by TIFF Technical Note #2) can be used for "high end" applications
that need to record a lot of additional data about an image.
TO DO
=====
Please send bug reports, offers of help, etc. to [email protected].
lib/libjpeg-turbo/README.md
Executable
+375
View File
@@ -0,0 +1,375 @@
Background
==========
libjpeg-turbo is a JPEG image codec that uses SIMD instructions to accelerate
baseline JPEG compression and decompression on x86, x86-64, Arm, PowerPC, and
MIPS systems, as well as progressive JPEG compression on x86, x86-64, and Arm
systems. On such systems, libjpeg-turbo is generally 2-6x as fast as libjpeg,
all else being equal. On other types of systems, libjpeg-turbo can still
outperform libjpeg by a significant amount, by virtue of its highly-optimized
Huffman coding routines. In many cases, the performance of libjpeg-turbo
rivals that of proprietary high-speed JPEG codecs.
libjpeg-turbo implements both the traditional libjpeg API as well as the less
powerful but more straightforward TurboJPEG API. libjpeg-turbo also features
colorspace extensions that allow it to compress from/decompress to 32-bit and
big-endian pixel buffers (RGBX, XBGR, etc.), as well as a full-featured Java
interface.
libjpeg-turbo was originally based on libjpeg/SIMD, an MMX-accelerated
derivative of libjpeg v6b developed by Miyasaka Masaru. The TigerVNC and
VirtualGL projects made numerous enhancements to the codec in 2009, and in
early 2010, libjpeg-turbo spun off into an independent project, with the goal
of making high-speed JPEG compression/decompression technology available to a
broader range of users and developers. libjpeg-turbo is an ISO/IEC and ITU-T
reference implementation of the JPEG standard.
More information about libjpeg-turbo can be found at
<https://libjpeg-turbo.org>.
Funding
=======
libjpeg-turbo is an independent open source project, but we rely on patronage
and funded development in order to maintain that independence. The easiest way
to ensure that libjpeg-turbo remains community-focused and free of any one
organization's agenda is to
[sponsor our project through GitHub](https://github.com/sponsors/libjpeg-turbo).
All sponsorship money goes directly toward funding the labor necessary to
maintain libjpeg-turbo, support the user community, and implement bug fixes and
strategically important features.
[![Sponsor libjpeg-turbo](https://img.shields.io/github/sponsors/libjpeg-turbo?label=Sponsor&logo=GitHub)](https://github.com/sponsors/libjpeg-turbo)
License
=======
libjpeg-turbo is covered by three compatible BSD-style open source licenses.
Refer to [LICENSE.md](LICENSE.md) for a roll-up of license terms.
Building libjpeg-turbo
======================
Refer to [BUILDING.md](BUILDING.md) for complete instructions.
Using libjpeg-turbo
===================
libjpeg-turbo includes two APIs that can be used to compress and decompress
JPEG images:
- **TurboJPEG API**<br>
This API provides an easy-to-use interface for compressing and decompressing
JPEG images in memory. It also provides some functionality that would not be
straightforward to achieve using the underlying libjpeg API, such as
generating planar YUV images and performing multiple simultaneous lossless
transforms on an image. The Java interface for libjpeg-turbo is written on
top of the TurboJPEG API. The TurboJPEG API is recommended for first-time
users of libjpeg-turbo. Refer to [tjcomp.c](src/tjcomp.c),
[tjdecomp.c](src/tjdecomp.c), [tjtran.c](src/tjtran.c),
[TJComp.java](java/TJComp.java), [TJDecomp.java](java/TJDecomp.java), and
[TJTran.java](java/TJTran.java) for examples of its usage and to
<https://libjpeg-turbo.org/Documentation/Documentation> for API
documentation.
- **libjpeg API**<br>
This is the de facto industry-standard API for compressing and decompressing
JPEG images. It is more difficult to use than the TurboJPEG API but also
more powerful. The libjpeg API implementation in libjpeg-turbo is both
API/ABI-compatible and mathematically compatible with libjpeg v6b. It can
also optionally be configured to be API/ABI-compatible with libjpeg v7 and v8
(see below.) Refer to [cjpeg.c](src/cjpeg.c) and [djpeg.c](src/djpeg.c) for
examples of its usage and to [libjpeg.txt](doc/libjpeg.txt) for API
documentation.
There is no significant performance advantage to either API when both are used
to perform similar operations.
Colorspace Extensions
---------------------
libjpeg-turbo includes extensions that allow JPEG images to be compressed
directly from (and decompressed directly to) buffers that use BGR, BGRX,
RGBX, XBGR, and XRGB pixel ordering. This is implemented with ten new
colorspace constants:
JCS_EXT_RGB /* red/green/blue */
JCS_EXT_RGBX /* red/green/blue/x */
JCS_EXT_BGR /* blue/green/red */
JCS_EXT_BGRX /* blue/green/red/x */
JCS_EXT_XBGR /* x/blue/green/red */
JCS_EXT_XRGB /* x/red/green/blue */
JCS_EXT_RGBA /* red/green/blue/alpha */
JCS_EXT_BGRA /* blue/green/red/alpha */
JCS_EXT_ABGR /* alpha/blue/green/red */
JCS_EXT_ARGB /* alpha/red/green/blue */
Setting `cinfo.in_color_space` (compression) or `cinfo.out_color_space`
(decompression) to one of these values will cause libjpeg-turbo to read the
red, green, and blue values from (or write them to) the appropriate position in
the pixel when compressing from/decompressing to an RGB buffer.
Your application can check for the existence of these extensions at compile
time with:
#ifdef JCS_EXTENSIONS
At run time, attempting to use these extensions with a libjpeg implementation
that does not support them will result in a "Bogus input colorspace" error.
Applications can trap this error in order to test whether run-time support is
available for the colorspace extensions.
When using the RGBX, BGRX, XBGR, and XRGB colorspaces during decompression, the
X byte is undefined, and in order to ensure the best performance, libjpeg-turbo
can set that byte to whatever value it wishes. If an application expects the X
byte to be used as an alpha channel, then it should specify `JCS_EXT_RGBA`,
`JCS_EXT_BGRA`, `JCS_EXT_ABGR`, or `JCS_EXT_ARGB`. When these colorspace
constants are used, the X byte is guaranteed to be 0xFF, which is interpreted
as opaque.
Your application can check for the existence of the alpha channel colorspace
extensions at compile time with:
#ifdef JCS_ALPHA_EXTENSIONS
[jcstest.c](src/jcstest.c), located in the libjpeg-turbo source tree,
demonstrates how to check for the existence of the colorspace extensions at
compile time and run time.
libjpeg v7 and v8 API/ABI Emulation
-----------------------------------
With libjpeg v7 and v8, new features were added that necessitated extending the
compression and decompression structures. Unfortunately, due to the exposed
nature of those structures, extending them also necessitated breaking backward
ABI compatibility with previous libjpeg releases. Thus, programs that were
built to use libjpeg v7 or v8 did not work with libjpeg-turbo, since it is
based on the libjpeg v6b code base. Although libjpeg v7 and v8 are not
as widely used as v6b, enough programs (including a few Linux distros) made
the switch that there was a demand to emulate the libjpeg v7 and v8 ABIs
in libjpeg-turbo. It should be noted, however, that this feature was added
primarily so that applications that had already been compiled to use libjpeg
v7+ could take advantage of accelerated baseline JPEG encoding/decoding
without recompiling. libjpeg-turbo does not claim to support all of the
libjpeg v7+ features, nor to produce identical output to libjpeg v7+ in all
cases (see below.)
By passing an argument of `-DWITH_JPEG7=1` or `-DWITH_JPEG8=1` to `cmake`, you
can build a version of libjpeg-turbo that emulates the libjpeg v7 or v8 ABI, so
that programs that are built against libjpeg v7 or v8 can be run with
libjpeg-turbo. The following section describes which libjpeg v7+ features are
supported and which aren't.
### Support for libjpeg v7 and v8 Features
#### Fully supported
- **libjpeg API: IDCT scaling extensions in decompressor**<br>
libjpeg-turbo supports IDCT scaling with scaling factors of 1/8, 1/4, 3/8,
1/2, 5/8, 3/4, 7/8, 9/8, 5/4, 11/8, 3/2, 13/8, 7/4, 15/8, and 2/1 (only 1/4
and 1/2 are SIMD-accelerated.)
- **libjpeg API: Arithmetic coding**
- **libjpeg API: In-memory source and destination managers**<br>
See notes below.
- **cjpeg: Separate quality settings for luminance and chrominance**<br>
Note that the libpjeg v7+ API was extended to accommodate this feature only
for convenience purposes. It has always been possible to implement this
feature with libjpeg v6b (see rdswitch.c for an example.)
- **cjpeg: 32-bit BMP support**
- **cjpeg: `-rgb` option**
- **jpegtran: Lossless cropping**
- **jpegtran: `-perfect` option**
- **jpegtran: Forcing width/height when performing lossless crop**
- **rdjpgcom: `-raw` option**
- **rdjpgcom: Locale awareness**
#### Not supported
NOTE: As of this writing, extensive research has been conducted into the
usefulness of DCT scaling as a means of data reduction and SmartScale as a
means of quality improvement. Readers are invited to peruse the research at
<https://libjpeg-turbo.org/About/SmartScale> and draw their own conclusions,
but it is the general belief of our project that these features have not
demonstrated sufficient usefulness to justify inclusion in libjpeg-turbo.
- **libjpeg API: DCT scaling in compressor**<br>
`cinfo.scale_num` and `cinfo.scale_denom` are silently ignored.
There is no technical reason why DCT scaling could not be supported when
emulating the libjpeg v7+ API/ABI, but without the SmartScale extension (see
below), only scaling factors of 1/2, 8/15, 4/7, 8/13, 2/3, 8/11, 4/5, and
8/9 would be available, which is of limited usefulness.
- **libjpeg API: SmartScale**<br>
`cinfo.block_size` is silently ignored.
SmartScale is an extension to the JPEG format that allows for DCT block
sizes other than 8x8. Providing support for this new format would be
feasible (particularly without full acceleration.) However, until/unless
the format becomes either an official industry standard or, at minimum, an
accepted solution in the community, we are hesitant to implement it, as
there is no sense of whether or how it might change in the future. It is
our belief that SmartScale has not demonstrated sufficient usefulness as a
lossless format nor as a means of quality enhancement, and thus our primary
interest in providing this feature would be as a means of supporting
additional DCT scaling factors.
- **libjpeg API: Fancy downsampling in compressor**<br>
`cinfo.do_fancy_downsampling` is silently ignored.
This requires the DCT scaling feature, which is not supported.
- **jpegtran: Scaling**<br>
This requires both the DCT scaling and SmartScale features, which are not
supported.
- **Lossless RGB JPEG files**<br>
This requires the SmartScale feature, which is not supported.
### What About libjpeg v9?
libjpeg v9 introduced yet another field to the JPEG compression structure
(`color_transform`), thus making the ABI backward incompatible with that of
libjpeg v8. This new field was introduced solely for the purpose of supporting
lossless SmartScale encoding. Furthermore, there was actually no reason to
extend the API in this manner, as the color transform could have just as easily
been activated by way of a new JPEG colorspace constant, thus preserving
backward ABI compatibility.
Our research (see link above) has shown that lossless SmartScale does not
generally accomplish anything that can't already be accomplished better with
existing, standard lossless formats. Therefore, at this time it is our belief
that there is not sufficient technical justification for software projects to
upgrade from libjpeg v8 to libjpeg v9, and thus there is not sufficient
technical justification for us to emulate the libjpeg v9 ABI.
In-Memory Source/Destination Managers
-------------------------------------
By default, libjpeg-turbo 1.3 and later includes the `jpeg_mem_src()` and
`jpeg_mem_dest()` functions, even when not emulating the libjpeg v8 API/ABI.
Previously, it was necessary to build libjpeg-turbo from source with libjpeg v8
API/ABI emulation in order to use the in-memory source/destination managers,
but several projects requested that those functions be included when emulating
the libjpeg v6b API/ABI as well. This allows the use of those functions by
programs that need them, without breaking ABI compatibility for programs that
don't, and it allows those functions to be provided in the "official"
libjpeg-turbo binaries.
Note that, on most Un*x systems, the dynamic linker will not look for a
function in a library until that function is actually used. Thus, if a program
is built against libjpeg-turbo 1.3+ and uses `jpeg_mem_src()` or
`jpeg_mem_dest()`, that program will not fail if run against an older version
of libjpeg-turbo or against libjpeg v7- until the program actually tries to
call `jpeg_mem_src()` or `jpeg_mem_dest()`. Such is not the case on Windows.
If a program is built against the libjpeg-turbo 1.3+ DLL and uses
`jpeg_mem_src()` or `jpeg_mem_dest()`, then it must use the libjpeg-turbo 1.3+
DLL at run time.
Both cjpeg and djpeg have been extended to allow testing the in-memory
source/destination manager functions. See their respective man pages for more
details.
Mathematical Compatibility
==========================
For the most part, libjpeg-turbo should produce identical output to libjpeg
v6b. There are two exceptions:
1. When decompressing a JPEG image that uses 4:4:0 chrominance subsampling, the
outputs of libjpeg v6b and libjpeg-turbo can differ because libjpeg-turbo
implements a "fancy" (smooth) 4:4:0 upsampling algorithm and libjpeg did not.
2. When using the floating point DCT/IDCT, the outputs of libjpeg v6b and
libjpeg-turbo can differ for the following reasons:
- The SSE/SSE2 floating point DCT implementation in libjpeg-turbo is ever
so slightly more accurate than the implementation in libjpeg v6b, but not
by any amount perceptible to human vision (generally in the range of 0.01
to 0.08 dB gain in PNSR.)
- When not using the SIMD extensions, libjpeg-turbo uses the more accurate
(and slightly faster) floating point IDCT algorithm introduced in libjpeg
v8a as opposed to the algorithm used in libjpeg v6b. It should be noted,
however, that this algorithm basically brings the accuracy of the
floating point IDCT in line with the accuracy of the accurate integer
IDCT. The floating point DCT/IDCT algorithms are mainly a legacy
feature, and they do not produce significantly more accuracy than the
accurate integer algorithms. (To put numbers on this, the typical
difference in PNSR between the two algorithms is less than 0.10 dB,
whereas changing the quality level by 1 in the upper range of the quality
scale is typically more like a 1.0 dB difference.)
- If the floating point algorithms in libjpeg-turbo are not implemented
using SIMD instructions on a particular platform, then the accuracy of
the floating point DCT/IDCT can depend on the compiler settings.
While libjpeg-turbo does emulate the libjpeg v8 API/ABI, under the hood it is
still using the same algorithms as libjpeg v6b, so there are several specific
cases in which libjpeg-turbo cannot be expected to produce the same output as
libjpeg v8:
- When decompressing using scaling factors of 1/2 and 1/4, because libjpeg v8
implements those scaling algorithms differently than libjpeg v6b does, and
libjpeg-turbo's SIMD extensions are based on the libjpeg v6b behavior.
- When using chrominance subsampling, because libjpeg v8 implements this
with its DCT/IDCT scaling algorithms rather than with a separate
downsampling/upsampling algorithm. In our testing, the subsampled/upsampled
output of libjpeg v8 is less accurate than that of libjpeg v6b for this
reason.
- When decompressing using a scaling factor > 1 and merged (AKA "non-fancy" or
"non-smooth") chrominance upsampling, because libjpeg v8 does not support
merged upsampling with scaling factors > 1.
Performance Pitfalls
====================
Restart Markers
---------------
The optimized Huffman decoder in libjpeg-turbo does not handle restart markers
in a way that makes the rest of the libjpeg infrastructure happy, so it is
necessary to use the slow Huffman decoder when decompressing a JPEG image that
has restart markers. This can cause the decompression performance to drop by
as much as 20%, but the performance will still be much greater than that of
libjpeg. Many consumer packages, such as Photoshop, use restart markers when
generating JPEG images, so images generated by those programs will experience
this issue.
Fast Integer Forward DCT at High Quality Levels
-----------------------------------------------
The algorithm used by the SIMD-accelerated quantization function cannot produce
correct results whenever the fast integer forward DCT is used along with a JPEG
quality of 98-100. Thus, libjpeg-turbo must use the non-SIMD quantization
function in those cases. This causes performance to drop by as much as 40%.
It is therefore strongly advised that you use the accurate integer forward DCT
whenever encoding images with a JPEG quality of 98 or higher.
Memory Debugger Pitfalls
========================
Valgrind and Memory Sanitizer (MSan) can generate false positives
(specifically, incorrect reports of uninitialized memory accesses) when used
with libjpeg-turbo's SIMD extensions. It is generally recommended that the
SIMD extensions be disabled, either by passing an argument of `-DWITH_SIMD=0`
to `cmake` when configuring the build or by setting the environment variable
`JSIMD_FORCENONE` to `1` at run time, when testing libjpeg-turbo with Valgrind,
MSan, or other memory debuggers.
lib/libjpeg-turbo/build_cross/cdjpeg.o
Executable
BIN
View File
Binary file not shown.
lib/libjpeg-turbo/build_cross/cjpeg.o
Executable
BIN
View File
Binary file not shown.
lib/libjpeg-turbo/build_cross/djpeg.o
Executable
BIN
View File
Binary file not shown.
lib/libjpeg-turbo/build_cross/example.o
Executable
BIN
View File
Binary file not shown.
lib/libjpeg-turbo/build_cross/jaricom.o
Executable
BIN
View File
Binary file not shown.
lib/libjpeg-turbo/build_cross/jcapimin.o
Executable
BIN
View File
Binary file not shown.
lib/libjpeg-turbo/build_cross/jcapistd.o
Executable
BIN
View File
Binary file not shown.
lib/libjpeg-turbo/build_cross/jcarith.o
Executable
BIN
View File
Binary file not shown.
lib/libjpeg-turbo/build_cross/jccoefct.o
Executable
BIN
View File
Binary file not shown.
lib/libjpeg-turbo/fuzz/CMakeLists.txt
Executable
+71
View File
@@ -0,0 +1,71 @@
if(NOT ENABLE_STATIC)
message(FATAL_ERROR "Fuzz targets require static libraries.")
endif()
if(NOT WITH_TURBOJPEG)
message(FATAL_ERROR "Fuzz targets require the TurboJPEG API library.")
endif()
set(FUZZ_BINDIR "" CACHE PATH
"Directory into which fuzz targets should be installed")
if(NOT FUZZ_BINDIR)
message(FATAL_ERROR "FUZZ_BINDIR must be specified.")
endif()
message(STATUS "FUZZ_BINDIR = ${FUZZ_BINDIR}")
set(FUZZ_LIBRARY "" CACHE STRING
"Path to fuzzer library or flags necessary to link with it")
if(NOT FUZZ_LIBRARY)
message(FATAL_ERROR "FUZZ_LIBRARY must be specified.")
endif()
message(STATUS "FUZZ_LIBRARY = ${FUZZ_LIBRARY}")
enable_language(CXX)
set(EFFECTIVE_CXX_FLAGS
"${CMAKE_CXX_FLAGS} ${CMAKE_CXX_FLAGS_${CMAKE_BUILD_TYPE_UC}}")
message(STATUS "C++ Compiler flags = ${EFFECTIVE_CXX_FLAGS}")
add_executable(cjpeg_fuzzer${FUZZER_SUFFIX} cjpeg.cc ../src/cdjpeg.c
../src/rdbmp.c ../src/rdgif.c ../src/wrapper/rdppm-8.c
../src/wrapper/rdppm-12.c ../src/wrapper/rdppm-16.c ../src/rdswitch.c
../src/rdtarga.c)
set_property(TARGET cjpeg_fuzzer${FUZZER_SUFFIX} PROPERTY COMPILE_FLAGS
${CDJPEG_COMPILE_FLAGS})
target_link_libraries(cjpeg_fuzzer${FUZZER_SUFFIX} ${FUZZ_LIBRARY} jpeg-static)
install(TARGETS cjpeg_fuzzer${FUZZER_SUFFIX}
RUNTIME DESTINATION ${FUZZ_BINDIR} COMPONENT bin)
macro(add_fuzz_target target source_file)
add_executable(${target}_fuzzer${FUZZER_SUFFIX} ${source_file})
target_link_libraries(${target}_fuzzer${FUZZER_SUFFIX} ${FUZZ_LIBRARY}
turbojpeg-static)
install(TARGETS ${target}_fuzzer${FUZZER_SUFFIX}
RUNTIME DESTINATION ${FUZZ_BINDIR} COMPONENT bin)
endmacro()
add_fuzz_target(compress compress.cc)
add_fuzz_target(compress_yuv compress_yuv.cc)
add_fuzz_target(compress_lossless compress_lossless.cc)
add_fuzz_target(compress12 compress12.cc)
add_fuzz_target(compress12_lossless compress12_lossless.cc)
add_fuzz_target(compress16_lossless compress16_lossless.cc)
# NOTE: This target is named libjpeg_turbo_fuzzer instead of decompress_fuzzer
# in order to preserve the corpora from Google's OSS-Fuzz target for
# libjpeg-turbo, which this target replaces.
add_fuzz_target(libjpeg_turbo decompress.cc)
add_executable(decompress_libjpeg_fuzzer${FUZZER_SUFFIX} decompress_libjpeg.cc)
target_link_libraries(decompress_libjpeg_fuzzer${FUZZER_SUFFIX} ${FUZZ_LIBRARY}
jpeg-static)
install(TARGETS decompress_libjpeg_fuzzer${FUZZER_SUFFIX}
RUNTIME DESTINATION ${FUZZ_BINDIR} COMPONENT bin)
add_fuzz_target(decompress_yuv decompress_yuv.cc)
add_fuzz_target(transform transform.cc)
lib/libjpeg-turbo/fuzz/build.sh
Executable
+44
View File
@@ -0,0 +1,44 @@
#!/bin/bash
set -u
set -e
FUZZER_SUFFIX=
if [ $# -ge 1 ]; then
FUZZER_SUFFIX="$1"
FUZZER_SUFFIX="`echo $1 | sed 's/\./_/g'`"
fi
if [ "$SANITIZER" = "memory" ]; then
export CFLAGS="$CFLAGS -DZERO_BUFFERS=1"
fi
cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_STATIC=1 -DENABLE_SHARED=0 \
-DCMAKE_C_FLAGS_RELWITHDEBINFO="-g -DNDEBUG" \
-DCMAKE_CXX_FLAGS_RELWITHDEBINFO="-g -DNDEBUG" -DCMAKE_INSTALL_PREFIX=$WORK \
-DWITH_FUZZ=1 -DFUZZ_BINDIR=$OUT -DFUZZ_LIBRARY=$LIB_FUZZING_ENGINE \
-DFUZZER_SUFFIX="$FUZZER_SUFFIX"
make "-j$(nproc)" "--load-average=$(nproc)"
make install
for fuzzer in cjpeg \
compress \
compress_yuv \
compress_lossless \
compress12 \
compress12_lossless \
compress16_lossless; do
cp $SRC/compress_fuzzer_seed_corpus.zip $OUT/${fuzzer}_fuzzer${FUZZER_SUFFIX}_seed_corpus.zip
done
FUZZ_DIR=$(dirname "$0")
for fuzzer in libjpeg_turbo \
decompress_libjpeg \
decompress_yuv \
transform; do
cp $SRC/decompress_fuzzer_seed_corpus.zip $OUT/${fuzzer}_fuzzer${FUZZER_SUFFIX}_seed_corpus.zip
if [ -f "$FUZZ_DIR/jpeg.dict" ]; then
cp "$FUZZ_DIR/jpeg.dict" $OUT/${fuzzer}_fuzzer${FUZZER_SUFFIX}.dict
fi
done
lib/libjpeg-turbo/fuzz/cjpeg.cc
Executable
+73
View File
@@ -0,0 +1,73 @@
/*
* Copyright (C) 2021, 2024, 2026 D. R. Commander. All Rights Reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* - Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* - Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
* - Neither the name of the libjpeg-turbo Project nor the names of its
* contributors may be used to endorse or promote products derived from this
* software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS",
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
/* This fuzz target wraps cjpeg in order to test esoteric compression options
as well as the GIF and Targa readers. */
#define CJPEG_FUZZER
extern "C" {
#include "../src/cjpeg.c"
}
#include <stdint.h>
#include <unistd.h>
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
char *argv1[] = {
(char *)"cjpeg", (char *)"-dct", (char *)"float", (char *)"-memdst",
(char *)"-quality", (char *)"100,99,98",
(char *)"-sample", (char *)"4x1,2x2,1x2", (char *)"-targa"
};
char *argv2[] = {
(char *)"cjpeg", (char *)"-dct", (char *)"float", (char *)"-memdst",
(char *)"-quality", (char *)"90,80,70", (char *)"-smooth", (char *)"50",
(char *)"-targa"
};
FILE *file = NULL;
if ((file = fmemopen((void *)data, size, "r")) == NULL)
goto bailout;
fseek(file, 0, SEEK_SET);
cjpeg_fuzzer(9, argv1, file);
fseek(file, 0, SEEK_SET);
cjpeg_fuzzer(9, argv2, file);
argv1[8] = argv2[8] = NULL;
fseek(file, 0, SEEK_SET);
cjpeg_fuzzer(8, argv1, file);
fseek(file, 0, SEEK_SET);
cjpeg_fuzzer(8, argv2, file);
bailout:
if (file) fclose(file);
return 0;
}
lib/libjpeg-turbo/fuzz/compress.cc
Executable
+162
View File
@@ -0,0 +1,162 @@
/*
* Copyright (C) 2021, 2023-2026 D. R. Commander. All Rights Reserved.
* Copyright (C) 2025 Leslie P. Polzer. All Rights Reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* - Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* - Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
* - Neither the name of the libjpeg-turbo Project nor the names of its
* contributors may be used to endorse or promote products derived from this
* software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS",
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include "../src/turbojpeg.h"
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <string.h>
#include <unistd.h>
extern "C" unsigned char *
_tj3LoadImageFromFileHandle8(tjhandle handle, FILE *file, int *width,
int align, int *height, int *pixelFormat);
#define NUMTESTS 7
struct test {
int bottomUp;
enum TJPF pf;
int colorspace;
enum TJSAMP subsamp;
int fastDCT, quality, optimize, progressive, arithmetic, noRealloc,
restartRows;
};
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
tjhandle handle = NULL;
unsigned char *imgBuf = NULL, *srcBuf, *dstBuf = NULL;
int width = 0, height = 0, ti;
FILE *file = NULL;
struct test tests[NUMTESTS] = {
/*
BU Pixel JPEG Subsampling Fst Qual Opt Prg Ari No Rst
Format Colorspace Level DCT Realc Rows */
{ 1, TJPF_RGB, TJCS_RGB, TJSAMP_444, 0, 100, 0, 0, 0, 0, 2 },
{ 0, TJPF_BGR, TJCS_YCbCr, TJSAMP_422, 0, 90, 0, 1, 0, 0, 0 },
{ 0, TJPF_RGBX, TJCS_YCbCr, TJSAMP_420, 1, 75, 0, 0, 1, 1, 0 },
{ 0, TJPF_BGRA, TJCS_YCbCr, TJSAMP_411, 0, 50, 0, 1, 1, 0, 0 },
{ 0, TJPF_XRGB, TJCS_GRAY, TJSAMP_GRAY, 0, 25, 0, 0, 0, 0, 0 },
{ 0, TJPF_GRAY, TJCS_GRAY, TJSAMP_GRAY, 0, 10, 0, 0, 0, 0, 0 },
{ 0, TJPF_CMYK, TJCS_YCCK, TJSAMP_440, 0, 1, 1, 0, 0, 0, 2 }
};
if ((file = fmemopen((void *)data, size, "r")) == NULL)
goto bailout;
if ((handle = tj3Init(TJINIT_COMPRESS)) == NULL)
goto bailout;
for (ti = 0; ti < NUMTESTS; ti++) {
int pf = tests[ti].pf;
size_t dstSize = 0, maxBufSize, i, sum = 0;
/* Test non-default compression options on specific iterations. */
tj3Set(handle, TJPARAM_BOTTOMUP, tests[ti].bottomUp);
tj3Set(handle, TJPARAM_COLORSPACE, tests[ti].colorspace);
tj3Set(handle, TJPARAM_FASTDCT, tests[ti].fastDCT);
tj3Set(handle, TJPARAM_OPTIMIZE, tests[ti].optimize);
tj3Set(handle, TJPARAM_PROGRESSIVE, tests[ti].progressive);
tj3Set(handle, TJPARAM_ARITHMETIC, tests[ti].arithmetic);
tj3Set(handle, TJPARAM_NOREALLOC, tests[ti].noRealloc);
tj3Set(handle, TJPARAM_RESTARTROWS, tests[ti].restartRows);
tj3Set(handle, TJPARAM_MAXPIXELS, 1048576);
/* tj3LoadImage8() will refuse to load images larger than 1 Megapixel, so
we don't need to check the width and height here. */
fseek(file, 0, SEEK_SET);
if ((imgBuf = _tj3LoadImageFromFileHandle8(handle, file, &width, 1,
&height, &pf)) == NULL) {
if (size < 2)
continue;
/* Derive image dimensions from input data. Use first 2 bytes to
influence width/height. */
width = (data[0] % 64) + 8; /* 8-71 */
height = (data[1] % 64) + 8; /* 8-71 */
size_t required_size = 2 + (size_t)width * height *
tjPixelSize[tests[ti].pf];
if (size < required_size) {
/* Not enough data - try smaller dimensions */
width = 8;
height = 8;
required_size = 2 + (size_t)width * height *
tjPixelSize[tests[ti].pf];
if (size < required_size)
continue;
}
/* Skip header bytes. */
srcBuf = (unsigned char *)data + 2;
} else
srcBuf = imgBuf;
dstSize = maxBufSize = tj3JPEGBufSize(width, height, tests[ti].subsamp);
if (tj3Get(handle, TJPARAM_NOREALLOC)) {
if ((dstBuf = (unsigned char *)tj3Alloc(dstSize)) == NULL)
goto bailout;
} else
dstBuf = NULL;
if (size >= 34)
tj3SetICCProfile(handle, (unsigned char *)&data[2], 32);
tj3Set(handle, TJPARAM_SUBSAMP, tests[ti].subsamp);
tj3Set(handle, TJPARAM_QUALITY, tests[ti].quality);
if (tj3Compress8(handle, srcBuf, width, 0, height, pf, &dstBuf,
&dstSize) == 0) {
/* Touch all of the output data in order to catch uninitialized reads
when using MemorySanitizer. */
for (i = 0; i < dstSize; i++)
sum += dstBuf[i];
}
tj3Free(dstBuf);
dstBuf = NULL;
tj3Free(imgBuf);
imgBuf = NULL;
/* Prevent the sum above from being optimized out. This test should never
be true, but the compiler doesn't know that. */
if (sum > 255 * maxBufSize)
goto bailout;
}
bailout:
tj3Free(dstBuf);
tj3Free(imgBuf);
if (file) fclose(file);
tj3Destroy(handle);
return 0;
}
lib/libjpeg-turbo/fuzz/compress12.cc
Executable
+161
View File
@@ -0,0 +1,161 @@
/*
* Copyright (C) 2021, 2023-2026 D. R. Commander. All Rights Reserved.
* Copyright (C) 2025 Leslie P. Polzer. All Rights Reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* - Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* - Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
* - Neither the name of the libjpeg-turbo Project nor the names of its
* contributors may be used to endorse or promote products derived from this
* software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS",
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include "../src/turbojpeg.h"
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <string.h>
#include <unistd.h>
extern "C" short *
_tj3LoadImageFromFileHandle12(tjhandle handle, FILE *file, int *width,
int align, int *height, int *pixelFormat);
#define NUMTESTS 7
struct test {
int bottomUp;
enum TJPF pf;
int colorspace;
enum TJSAMP subsamp;
int fastDCT, quality, progressive, arithmetic, noRealloc, restartRows;
};
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
tjhandle handle = NULL;
short *imgBuf = NULL, *srcBuf;
unsigned char *dstBuf = NULL;
int width = 0, height = 0, ti;
FILE *file = NULL;
struct test tests[NUMTESTS] = {
/*
BU Pixel JPEG Subsampling Fst Qual Prg Ari No Rst
Format Colorspace Level DCT Realc Rows */
{ 0, TJPF_RGB, TJCS_YCbCr, TJSAMP_444, 1, 100, 0, 0, 1, 0 },
{ 0, TJPF_BGR, TJCS_YCbCr, TJSAMP_422, 0, 90, 0, 0, 0, 0 },
{ 0, TJPF_RGBX, TJCS_RGB, TJSAMP_420, 0, 75, 0, 1, 0, 1 },
{ 0, TJPF_BGRA, TJCS_YCbCr, TJSAMP_411, 0, 50, 0, 0, 0, 0 },
{ 0, TJPF_XRGB, TJCS_GRAY, TJSAMP_GRAY, 0, 25, 0, 0, 0, 0 },
{ 0, TJPF_GRAY, TJCS_GRAY, TJSAMP_GRAY, 0, 10, 1, 0, 0, 0 },
{ 1, TJPF_CMYK, TJCS_YCCK, TJSAMP_440, 0, 1, 1, 1, 0, 1 }
};
if ((file = fmemopen((void *)data, size, "r")) == NULL)
goto bailout;
if ((handle = tj3Init(TJINIT_COMPRESS)) == NULL)
goto bailout;
for (ti = 0; ti < NUMTESTS; ti++) {
int pf = tests[ti].pf;
size_t dstSize = 0, maxBufSize, i, sum = 0;
/* Test non-default compression options on specific iterations. */
tj3Set(handle, TJPARAM_BOTTOMUP, tests[ti].bottomUp);
tj3Set(handle, TJPARAM_COLORSPACE, tests[ti].colorspace);
tj3Set(handle, TJPARAM_FASTDCT, tests[ti].fastDCT);
tj3Set(handle, TJPARAM_PROGRESSIVE, tests[ti].progressive);
tj3Set(handle, TJPARAM_ARITHMETIC, tests[ti].arithmetic);
tj3Set(handle, TJPARAM_NOREALLOC, tests[ti].noRealloc);
tj3Set(handle, TJPARAM_RESTARTROWS, tests[ti].restartRows);
tj3Set(handle, TJPARAM_MAXPIXELS, 1048576);
/* tj3LoadImage12() will refuse to load images larger than 1 Megapixel, so
we don't need to check the width and height here. */
fseek(file, 0, SEEK_SET);
if ((imgBuf = _tj3LoadImageFromFileHandle12(handle, file, &width, 1,
&height, &pf)) == NULL) {
if (size < 2)
continue;
/* Derive image dimensions from input data. Use first 2 bytes to
influence width/height. */
width = (data[0] % 64) + 8; /* 8-71 */
height = (data[1] % 64) + 8; /* 8-71 */
size_t required_size = 2 + (size_t)width * height *
tjPixelSize[tests[ti].pf] * 2;
if (size < required_size) {
/* Not enough data - try smaller dimensions */
width = 8;
height = 8;
required_size = 2 + (size_t)width * height *
tjPixelSize[tests[ti].pf] * 2;
if (size < required_size)
continue;
}
/* Skip header bytes. */
srcBuf = (short *)(data + 2);
} else
srcBuf = imgBuf;
dstSize = maxBufSize = tj3JPEGBufSize(width, height, tests[ti].subsamp);
if (tj3Get(handle, TJPARAM_NOREALLOC)) {
if ((dstBuf = (unsigned char *)tj3Alloc(dstSize)) == NULL)
goto bailout;
} else
dstBuf = NULL;
if (size >= 34)
tj3SetICCProfile(handle, (unsigned char *)&data[2], 32);
tj3Set(handle, TJPARAM_SUBSAMP, tests[ti].subsamp);
tj3Set(handle, TJPARAM_QUALITY, tests[ti].quality);
if (tj3Compress12(handle, srcBuf, width, 0, height, pf, &dstBuf,
&dstSize) == 0) {
/* Touch all of the output data in order to catch uninitialized reads
when using MemorySanitizer. */
for (i = 0; i < dstSize; i++)
sum += dstBuf[i];
}
tj3Free(dstBuf);
dstBuf = NULL;
tj3Free(imgBuf);
imgBuf = NULL;
/* Prevent the sum above from being optimized out. This test should never
be true, but the compiler doesn't know that. */
if (sum > 255 * maxBufSize)
goto bailout;
}
bailout:
tj3Free(dstBuf);
tj3Free(imgBuf);
if (file) fclose(file);
tj3Destroy(handle);
return 0;
}
lib/libjpeg-turbo/fuzz/compress12_lossless.cc
Executable
+157
View File
@@ -0,0 +1,157 @@
/*
* Copyright (C) 2021-2026 D. R. Commander. All Rights Reserved.
* Copyright (C) 2025 Leslie P. Polzer. All Rights Reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* - Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* - Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
* - Neither the name of the libjpeg-turbo Project nor the names of its
* contributors may be used to endorse or promote products derived from this
* software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS",
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include "../src/turbojpeg.h"
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <string.h>
#include <unistd.h>
extern "C" short *
_tj3LoadImageFromFileHandle12(tjhandle handle, FILE *file, int *width,
int align, int *height, int *pixelFormat);
#define NUMTESTS 7
struct test {
int bottomUp;
enum TJPF pf;
int precision, psv, pt, noRealloc, restartRows;
};
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
tjhandle handle = NULL;
short *imgBuf = NULL, *srcBuf;
unsigned char *dstBuf = NULL;
int width = 0, height = 0, ti;
FILE *file = NULL;
struct test tests[NUMTESTS] = {
/*
BU Pixel Data PSV Pt No Rst
Format Prec Realc Rows */
{ 1, TJPF_RGB, 12, 1, 0, 1, 1 },
{ 0, TJPF_BGR, 11, 2, 2, 1, 0 },
{ 0, TJPF_RGBX, 10, 3, 4, 0, 0 },
{ 0, TJPF_BGRA, 9, 4, 7, 1, 0 },
{ 0, TJPF_XRGB, 12, 5, 5, 1, 0 },
{ 0, TJPF_GRAY, 12, 6, 3, 1, 0 },
{ 0, TJPF_CMYK, 12, 7, 0, 1, 1 }
};
if ((file = fmemopen((void *)data, size, "r")) == NULL)
goto bailout;
if ((handle = tj3Init(TJINIT_COMPRESS)) == NULL)
goto bailout;
for (ti = 0; ti < NUMTESTS; ti++) {
int pf = tests[ti].pf;
size_t dstSize = 0, maxBufSize, i, sum = 0;
/* Test non-default compression options on specific iterations. */
tj3Set(handle, TJPARAM_BOTTOMUP, tests[ti].bottomUp);
tj3Set(handle, TJPARAM_NOREALLOC, tests[ti].noRealloc);
tj3Set(handle, TJPARAM_PRECISION, tests[ti].precision);
tj3Set(handle, TJPARAM_RESTARTROWS, tests[ti].restartRows);
tj3Set(handle, TJPARAM_MAXPIXELS, 1048576);
/* tj3LoadImage12() will refuse to load images larger than 1 Megapixel, so
we don't need to check the width and height here. */
fseek(file, 0, SEEK_SET);
if ((imgBuf = _tj3LoadImageFromFileHandle12(handle, file, &width, 1,
&height, &pf)) == NULL) {
if (size < 2)
continue;
/* Derive image dimensions from input data. Use first 2 bytes to
influence width/height. */
width = (data[0] % 64) + 8; /* 8-71 */
height = (data[1] % 64) + 8; /* 8-71 */
size_t required_size = 2 + (size_t)width * height *
tjPixelSize[tests[ti].pf] * 2;
if (size < required_size) {
/* Not enough data - try smaller dimensions */
width = 8;
height = 8;
required_size = 2 + (size_t)width * height *
tjPixelSize[tests[ti].pf] * 2;
if (size < required_size)
continue;
}
/* Skip header bytes. */
srcBuf = (short *)(data + 2);
} else
srcBuf = imgBuf;
dstSize = maxBufSize = tj3JPEGBufSize(width, height, TJSAMP_444);
if (tj3Get(handle, TJPARAM_NOREALLOC)) {
if ((dstBuf = (unsigned char *)tj3Alloc(dstSize)) == NULL)
goto bailout;
} else
dstBuf = NULL;
if (size >= 34)
tj3SetICCProfile(handle, (unsigned char *)&data[2], 32);
tj3Set(handle, TJPARAM_LOSSLESS, 1);
tj3Set(handle, TJPARAM_LOSSLESSPSV, tests[ti].psv);
tj3Set(handle, TJPARAM_LOSSLESSPT, tests[ti].pt);
if (tj3Compress12(handle, srcBuf, width, 0, height, pf, &dstBuf,
&dstSize) == 0) {
/* Touch all of the output data in order to catch uninitialized reads
when using MemorySanitizer. */
for (i = 0; i < dstSize; i++)
sum += dstBuf[i];
}
tj3Free(dstBuf);
dstBuf = NULL;
tj3Free(imgBuf);
imgBuf = NULL;
/* Prevent the sum above from being optimized out. This test should never
be true, but the compiler doesn't know that. */
if (sum > 255 * maxBufSize)
goto bailout;
}
bailout:
tj3Free(dstBuf);
tj3Free(imgBuf);
if (file) fclose(file);
tj3Destroy(handle);
return 0;
}
lib/libjpeg-turbo/fuzz/compress16_lossless.cc
Executable
+157
View File
@@ -0,0 +1,157 @@
/*
* Copyright (C) 2021-2026 D. R. Commander. All Rights Reserved.
* Copyright (C) 2025 Leslie P. Polzer. All Rights Reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* - Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* - Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
* - Neither the name of the libjpeg-turbo Project nor the names of its
* contributors may be used to endorse or promote products derived from this
* software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS",
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include "../src/turbojpeg.h"
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <string.h>
#include <unistd.h>
extern "C" unsigned short *
_tj3LoadImageFromFileHandle16(tjhandle handle, FILE *file, int *width,
int align, int *height, int *pixelFormat);
#define NUMTESTS 7
struct test {
int bottomUp;
enum TJPF pf;
int precision, psv, pt, noRealloc, restartRows;
};
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
tjhandle handle = NULL;
unsigned short *imgBuf = NULL, *srcBuf;
unsigned char *dstBuf = NULL;
int width = 0, height = 0, ti;
FILE *file = NULL;
struct test tests[NUMTESTS] = {
/*
BU Pixel Data PSV Pt No Rst
Format Prec Realc Rows */
{ 1, TJPF_RGB, 16, 1, 0, 1, 1 },
{ 0, TJPF_BGR, 15, 2, 2, 1, 0 },
{ 0, TJPF_RGBX, 14, 3, 4, 0, 0 },
{ 0, TJPF_BGRA, 13, 4, 7, 1, 0 },
{ 0, TJPF_XRGB, 16, 5, 5, 1, 0 },
{ 0, TJPF_GRAY, 16, 6, 3, 1, 0 },
{ 0, TJPF_CMYK, 16, 7, 0, 1, 1 }
};
if ((file = fmemopen((void *)data, size, "r")) == NULL)
goto bailout;
if ((handle = tj3Init(TJINIT_COMPRESS)) == NULL)
goto bailout;
for (ti = 0; ti < NUMTESTS; ti++) {
int pf = tests[ti].pf;
size_t dstSize = 0, maxBufSize, i, sum = 0;
/* Test non-default compression options on specific iterations. */
tj3Set(handle, TJPARAM_BOTTOMUP, tests[ti].bottomUp);
tj3Set(handle, TJPARAM_NOREALLOC, tests[ti].noRealloc);
tj3Set(handle, TJPARAM_PRECISION, tests[ti].precision);
tj3Set(handle, TJPARAM_RESTARTROWS, tests[ti].restartRows);
tj3Set(handle, TJPARAM_MAXPIXELS, 1048576);
/* tj3LoadImage16() will refuse to load images larger than 1 Megapixel, so
we don't need to check the width and height here. */
fseek(file, 0, SEEK_SET);
if ((imgBuf = _tj3LoadImageFromFileHandle16(handle, file, &width, 1,
&height, &pf)) == NULL) {
if (size < 2)
continue;
/* Derive image dimensions from input data. Use first 2 bytes to
influence width/height. */
width = (data[0] % 64) + 8; /* 8-71 */
height = (data[1] % 64) + 8; /* 8-71 */
size_t required_size = 2 + (size_t)width * height *
tjPixelSize[tests[ti].pf] * 2;
if (size < required_size) {
/* Not enough data - try smaller dimensions */
width = 8;
height = 8;
required_size = 2 + (size_t)width * height *
tjPixelSize[tests[ti].pf] * 2;
if (size < required_size)
continue;
}
/* Skip header bytes. */
srcBuf = (unsigned short *)(data + 2);
} else
srcBuf = imgBuf;
dstSize = maxBufSize = tj3JPEGBufSize(width, height, TJSAMP_444);
if (tj3Get(handle, TJPARAM_NOREALLOC)) {
if ((dstBuf = (unsigned char *)tj3Alloc(dstSize)) == NULL)
goto bailout;
} else
dstBuf = NULL;
if (size >= 34)
tj3SetICCProfile(handle, (unsigned char *)&data[2], 32);
tj3Set(handle, TJPARAM_LOSSLESS, 1);
tj3Set(handle, TJPARAM_LOSSLESSPSV, tests[ti].psv);
tj3Set(handle, TJPARAM_LOSSLESSPT, tests[ti].pt);
if (tj3Compress16(handle, srcBuf, width, 0, height, pf, &dstBuf,
&dstSize) == 0) {
/* Touch all of the output data in order to catch uninitialized reads
when using MemorySanitizer. */
for (i = 0; i < dstSize; i++)
sum += dstBuf[i];
}
tj3Free(dstBuf);
dstBuf = NULL;
tj3Free(imgBuf);
imgBuf = NULL;
/* Prevent the sum above from being optimized out. This test should never
be true, but the compiler doesn't know that. */
if (sum > 255 * maxBufSize)
goto bailout;
}
bailout:
tj3Free(dstBuf);
tj3Free(imgBuf);
if (file) fclose(file);
tj3Destroy(handle);
return 0;
}
lib/libjpeg-turbo/fuzz/compress_lossless.cc
Executable
+156
View File
@@ -0,0 +1,156 @@
/*
* Copyright (C) 2021-2026 D. R. Commander. All Rights Reserved.
* Copyright (C) 2025 Leslie P. Polzer. All Rights Reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* - Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* - Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
* - Neither the name of the libjpeg-turbo Project nor the names of its
* contributors may be used to endorse or promote products derived from this
* software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS",
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include "../src/turbojpeg.h"
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <string.h>
#include <unistd.h>
extern "C" unsigned char *
_tj3LoadImageFromFileHandle8(tjhandle handle, FILE *file, int *width,
int align, int *height, int *pixelFormat);
#define NUMTESTS 7
struct test {
int bottomUp;
enum TJPF pf;
int precision, psv, pt, noRealloc, restartRows;
};
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
tjhandle handle = NULL;
unsigned char *imgBuf = NULL, *srcBuf, *dstBuf = NULL;
int width = 0, height = 0, ti;
FILE *file = NULL;
struct test tests[NUMTESTS] = {
/*
BU Pixel Data PSV Pt No Rst
Format Prec Realc Rows */
{ 0, TJPF_RGB, 8, 1, 0, 1, 1 },
{ 0, TJPF_BGR, 7, 2, 5, 1, 0 },
{ 0, TJPF_RGBX, 6, 3, 4, 0, 0 },
{ 0, TJPF_BGRA, 5, 4, 1, 1, 0 },
{ 1, TJPF_XRGB, 4, 5, 3, 1, 0 },
{ 0, TJPF_GRAY, 3, 6, 2, 1, 0 },
{ 0, TJPF_CMYK, 2, 7, 0, 1, 1 }
};
if ((file = fmemopen((void *)data, size, "r")) == NULL)
goto bailout;
if ((handle = tj3Init(TJINIT_COMPRESS)) == NULL)
goto bailout;
for (ti = 0; ti < NUMTESTS; ti++) {
int pf = tests[ti].pf;
size_t dstSize = 0, maxBufSize, i, sum = 0;
/* Test non-default compression options on specific iterations. */
tj3Set(handle, TJPARAM_BOTTOMUP, tests[ti].bottomUp);
tj3Set(handle, TJPARAM_NOREALLOC, tests[ti].noRealloc);
tj3Set(handle, TJPARAM_PRECISION, tests[ti].precision);
tj3Set(handle, TJPARAM_RESTARTROWS, tests[ti].restartRows);
tj3Set(handle, TJPARAM_MAXPIXELS, 1048576);
/* tj3LoadImage8() will refuse to load images larger than 1 Megapixel, so
we don't need to check the width and height here. */
fseek(file, 0, SEEK_SET);
if ((imgBuf = _tj3LoadImageFromFileHandle8(handle, file, &width, 1,
&height, &pf)) == NULL) {
if (size < 2)
continue;
/* Derive image dimensions from input data. Use first 2 bytes to
influence width/height. */
width = (data[0] % 64) + 8; /* 8-71 */
height = (data[1] % 64) + 8; /* 8-71 */
size_t required_size = 2 + (size_t)width * height *
tjPixelSize[tests[ti].pf];
if (size < required_size) {
/* Not enough data - try smaller dimensions */
width = 8;
height = 8;
required_size = 2 + (size_t)width * height *
tjPixelSize[tests[ti].pf];
if (size < required_size)
continue;
}
/* Skip header bytes. */
srcBuf = (unsigned char *)data + 2;
} else
srcBuf = imgBuf;
dstSize = maxBufSize = tj3JPEGBufSize(width, height, TJSAMP_444);
if (tj3Get(handle, TJPARAM_NOREALLOC)) {
if ((dstBuf = (unsigned char *)tj3Alloc(dstSize)) == NULL)
goto bailout;
} else
dstBuf = NULL;
if (size >= 34)
tj3SetICCProfile(handle, (unsigned char *)&data[2], 32);
tj3Set(handle, TJPARAM_LOSSLESS, 1);
tj3Set(handle, TJPARAM_LOSSLESSPSV, tests[ti].psv);
tj3Set(handle, TJPARAM_LOSSLESSPT, tests[ti].pt);
if (tj3Compress8(handle, srcBuf, width, 0, height, pf, &dstBuf,
&dstSize) == 0) {
/* Touch all of the output data in order to catch uninitialized reads
when using MemorySanitizer. */
for (i = 0; i < dstSize; i++)
sum += dstBuf[i];
}
tj3Free(dstBuf);
dstBuf = NULL;
tj3Free(imgBuf);
imgBuf = NULL;
/* Prevent the sum above from being optimized out. This test should never
be true, but the compiler doesn't know that. */
if (sum > 255 * maxBufSize)
goto bailout;
}
bailout:
tj3Free(dstBuf);
tj3Free(imgBuf);
if (file) fclose(file);
tj3Destroy(handle);
return 0;
}
lib/libjpeg-turbo/fuzz/compress_yuv.cc
Executable
+161
View File
@@ -0,0 +1,161 @@
/*
* Copyright (C) 2021-2026 D. R. Commander. All Rights Reserved.
* Copyright (C) 2025 Leslie P. Polzer. All Rights Reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* - Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* - Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
* - Neither the name of the libjpeg-turbo Project nor the names of its
* contributors may be used to endorse or promote products derived from this
* software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS",
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include "../src/turbojpeg.h"
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <string.h>
#include <unistd.h>
extern "C" unsigned char *
_tj3LoadImageFromFileHandle8(tjhandle handle, FILE *file, int *width,
int align, int *height, int *pixelFormat);
#define NUMTESTS 6
struct test {
int bottomUp;
enum TJPF pf;
enum TJSAMP subsamp;
int fastDCT, quality, optimize, progressive, arithmetic, restartBlocks;
};
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
tjhandle handle = NULL;
unsigned char *imgBuf = NULL, *srcBuf, *dstBuf = NULL, *yuvBuf = NULL;
int width = 0, height = 0, ti;
FILE *file = NULL;
struct test tests[NUMTESTS] = {
/*
BU Pixel Subsampling Fst Qual Opt Prg Ari Rst
Format Level DCT Blks */
{ 0, TJPF_XBGR, TJSAMP_444, 1, 100, 0, 0, 0, 0 },
{ 0, TJPF_XRGB, TJSAMP_422, 0, 90, 0, 1, 0, 4 },
{ 0, TJPF_BGR, TJSAMP_420, 0, 75, 0, 0, 0, 0 },
{ 0, TJPF_RGB, TJSAMP_411, 0, 50, 1, 0, 0, 0 },
{ 0, TJPF_BGR, TJSAMP_GRAY, 0, 25, 0, 0, 1, 0 },
{ 1, TJPF_GRAY, TJSAMP_GRAY, 1, 10, 0, 1, 1, 4 }
};
if ((file = fmemopen((void *)data, size, "r")) == NULL)
goto bailout;
if ((handle = tj3Init(TJINIT_COMPRESS)) == NULL)
goto bailout;
for (ti = 0; ti < NUMTESTS; ti++) {
int pf = tests[ti].pf;
size_t dstSize = 0, maxBufSize, i, sum = 0;
/* Test non-default compression options on specific iterations. */
tj3Set(handle, TJPARAM_BOTTOMUP, tests[ti].bottomUp);
tj3Set(handle, TJPARAM_FASTDCT, tests[ti].fastDCT);
tj3Set(handle, TJPARAM_OPTIMIZE, tests[ti].optimize);
tj3Set(handle, TJPARAM_PROGRESSIVE, tests[ti].progressive);
tj3Set(handle, TJPARAM_ARITHMETIC, tests[ti].arithmetic);
tj3Set(handle, TJPARAM_NOREALLOC, 1);
tj3Set(handle, TJPARAM_RESTARTBLOCKS, tests[ti].restartBlocks);
tj3Set(handle, TJPARAM_MAXPIXELS, 1048576);
/* tj3LoadImage8() will refuse to load images larger than 1 Megapixel, so
we don't need to check the width and height here. */
fseek(file, 0, SEEK_SET);
if ((imgBuf = _tj3LoadImageFromFileHandle8(handle, file, &width, 1,
&height, &pf)) == NULL) {
if (size < 2)
continue;
/* Derive image dimensions from input data. Use first 2 bytes to
influence width/height. These must be multiples of the maximum iMCU
size for the subsampling levels we plan to test. */
width = ((data[0] % 4) + 1) * 32; /* 32-128, multiple of 32 */
height = ((data[1] % 8) + 1) * 16; /* 16-128, multiple of 16 */
size_t required_size = 2 + (size_t)width * height *
tjPixelSize[tests[ti].pf];
if (size < required_size) {
/* Not enough data - try smaller dimensions */
width = 32;
height = 16;
required_size = 2 + (size_t)width * height *
tjPixelSize[tests[ti].pf];
if (size < required_size)
continue;
}
/* Skip header bytes. */
srcBuf = (unsigned char *)data + 2;
} else
srcBuf = imgBuf;
dstSize = maxBufSize = tj3JPEGBufSize(width, height, tests[ti].subsamp);
if ((dstBuf = (unsigned char *)tj3Alloc(dstSize)) == NULL)
goto bailout;
if ((yuvBuf =
(unsigned char *)malloc(tj3YUVBufSize(width, 1, height,
tests[ti].subsamp))) == NULL)
goto bailout;
tj3Set(handle, TJPARAM_SUBSAMP, tests[ti].subsamp);
tj3Set(handle, TJPARAM_QUALITY, tests[ti].quality);
if (tj3EncodeYUV8(handle, srcBuf, width, 0, height, pf, yuvBuf, 1) == 0 &&
tj3CompressFromYUV8(handle, yuvBuf, width, 1, height, &dstBuf,
&dstSize) == 0) {
/* Touch all of the output data in order to catch uninitialized reads
when using MemorySanitizer. */
for (i = 0; i < dstSize; i++)
sum += dstBuf[i];
}
tj3Free(dstBuf);
dstBuf = NULL;
free(yuvBuf);
yuvBuf = NULL;
tj3Free(imgBuf);
imgBuf = NULL;
/* Prevent the sum above from being optimized out. This test should never
be true, but the compiler doesn't know that. */
if (sum > 255 * maxBufSize)
goto bailout;
}
bailout:
tj3Free(dstBuf);
free(yuvBuf);
tj3Free(imgBuf);
if (file) fclose(file);
tj3Destroy(handle);
return 0;
}
lib/libjpeg-turbo/fuzz/decompress.cc
Executable
+146
View File
@@ -0,0 +1,146 @@
/*
* Copyright (C) 2021-2026 D. R. Commander. All Rights Reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* - Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* - Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
* - Neither the name of the libjpeg-turbo Project nor the names of its
* contributors may be used to endorse or promote products derived from this
* software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS",
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include "../src/turbojpeg.h"
#include <stdlib.h>
#include <stdint.h>
#include <string.h>
#define NUMPF 5
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
tjhandle handle = NULL;
void *dstBuf = NULL;
int width = 0, height = 0, precision, sampleSize, pfi;
/* TJPF_RGB-TJPF_BGR share the same code paths, as do TJPF_RGBX-TJPF_XRGB and
TJPF_RGBA-TJPF_ARGB. Thus, the pixel formats below should be the minimum
necessary to achieve full coverage. */
enum TJPF pixelFormats[NUMPF] =
{ TJPF_RGB, TJPF_BGRX, TJPF_ABGR, TJPF_GRAY, TJPF_CMYK };
if ((handle = tj3Init(TJINIT_DECOMPRESS)) == NULL)
goto bailout;
/* We ignore the return value of tj3DecompressHeader(), because malformed
JPEG images that might expose issues in libjpeg-turbo might also have
header errors that cause tj3DecompressHeader() to fail. */
tj3DecompressHeader(handle, data, size);
width = tj3Get(handle, TJPARAM_JPEGWIDTH);
height = tj3Get(handle, TJPARAM_JPEGHEIGHT);
precision = tj3Get(handle, TJPARAM_PRECISION);
sampleSize = (precision > 8 ? 2 : 1);
/* Ignore 0-pixel images and images larger than 1 Megapixel, as Google's
OSS-Fuzz target for libjpeg-turbo did. Casting width to (uint64_t)
prevents integer overflow if width * height > INT_MAX. */
if (width < 1 || height < 1 || (uint64_t)width * height > 1048576)
goto bailout;
tj3Set(handle, TJPARAM_SCANLIMIT, 100);
for (pfi = 0; pfi < NUMPF; pfi++) {
int w = width, h = height;
int pf = pixelFormats[pfi], i;
int64_t sum = 0;
/* Test non-default decompression options on the first iteration. */
tj3Set(handle, TJPARAM_BOTTOMUP, pfi == 0);
tj3Set(handle, TJPARAM_FASTUPSAMPLE, pfi == 0);
if (!tj3Get(handle, TJPARAM_LOSSLESS)) {
tj3Set(handle, TJPARAM_FASTDCT, pfi == 0);
/* Test IDCT scaling on the second and third iterations. */
if (pfi == 1 || pfi == 2) {
tjscalingfactor sf = { 1, pfi == 1 ? 2 : 8 };
tj3SetScalingFactor(handle, sf);
w = TJSCALED(width, sf);
h = TJSCALED(height, sf);
} else
tj3SetScalingFactor(handle, TJUNSCALED);
/* Test partial image decompression on the second and fourth iterations,
if the image is large enough. */
if ((pfi == 1 || pfi == 3) && w >= 97 && h >= 75) {
tjregion cr = { 32, 16, 65, 59 };
tj3SetCroppingRegion(handle, cr);
} else
tj3SetCroppingRegion(handle, TJUNCROPPED);
}
if ((dstBuf = tj3Alloc(w * h * tjPixelSize[pf] * sampleSize)) == NULL)
goto bailout;
if (precision == 8) {
if (tj3Decompress8(handle, data, size, (unsigned char *)dstBuf, 0,
pf) == 0) {
/* Touch all of the output pixels in order to catch uninitialized reads
when using MemorySanitizer. */
for (i = 0; i < w * h * tjPixelSize[pf]; i++)
sum += ((unsigned char *)dstBuf)[i];
} else if (!strcmp(tj3GetErrorStr(handle),
"Progressive JPEG image has more than 100 scans"))
goto bailout;
} else if (precision == 12) {
if (tj3Decompress12(handle, data, size, (short *)dstBuf, 0, pf) == 0) {
/* Touch all of the output pixels in order to catch uninitialized reads
when using MemorySanitizer. */
for (i = 0; i < w * h * tjPixelSize[pf]; i++)
sum += ((short *)dstBuf)[i];
} else if (!strcmp(tj3GetErrorStr(handle),
"Progressive JPEG image has more than 100 scans"))
goto bailout;
} else {
if (tj3Decompress16(handle, data, size, (unsigned short *)dstBuf, 0,
pf) == 0) {
/* Touch all of the output pixels in order to catch uninitialized reads
when using MemorySanitizer. */
for (i = 0; i < w * h * tjPixelSize[pf]; i++)
sum += ((unsigned short *)dstBuf)[i];
} else if (!strcmp(tj3GetErrorStr(handle),
"Progressive JPEG image has more than 100 scans"))
goto bailout;
}
tj3Free(dstBuf);
dstBuf = NULL;
/* Prevent the sum above from being optimized out. This test should never
be true, but the compiler doesn't know that. */
if (sum > ((1LL << precision) - 1LL) * 1048576LL * tjPixelSize[pf])
goto bailout;
}
bailout:
tj3Free(dstBuf);
tj3Destroy(handle);
return 0;
}
lib/libjpeg-turbo/fuzz/decompress_libjpeg.cc
Executable
+291
View File
@@ -0,0 +1,291 @@
/*
* Copyright (C) 2021-2024, 2026 D. R. Commander. All Rights Reserved.
* Copyright (C) 2025 Leslie P. Polzer. All Rights Reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* - Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* - Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
* - Neither the name of the libjpeg-turbo Project nor the names of its
* contributors may be used to endorse or promote products derived from this
* software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS",
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
/* This fuzzer uses the libjpeg API to exercise code paths that are not covered
* by the other fuzzers (or by the TurboJPEG API in general):
*
* - JCS_UNKNOWN (NULL color conversion with a component count other than 3 or
* 4)
* - Floating point IDCT
* - Buffered-image mode
* - Interstitial line skipping
* - jpeg_save_markers() with a length limit
* - Custom marker processor
* - JCS_RGB565
* - Color quantization
*/
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <setjmp.h>
extern "C" {
#include "../src/jpeglib.h"
#include "../src/jerror.h"
}
struct fuzzer_error_mgr {
struct jpeg_error_mgr pub;
jmp_buf setjmp_buffer;
};
static void fuzzer_error_exit(j_common_ptr cinfo)
{
struct fuzzer_error_mgr *fuzz_err = (struct fuzzer_error_mgr *)cinfo->err;
longjmp(fuzz_err->setjmp_buffer, 1);
}
static void fuzzer_emit_message(j_common_ptr cinfo, int msg_level)
{
}
static int64_t marker_sum = 0;
static boolean custom_marker_processor(j_decompress_ptr cinfo)
{
struct jpeg_source_mgr *src = cinfo->src;
INT32 length;
/* Read and consume the 2-byte length field. */
if (src->bytes_in_buffer < 2)
return FALSE;
length = ((INT32)src->next_input_byte[0] << 8) +
(INT32)src->next_input_byte[1];
src->next_input_byte += 2;
src->bytes_in_buffer -= 2;
length -= 2;
if (length < 0)
return FALSE;
/* Consume and touch all marker data in order to catch uninitialized reads
when using MemorySanitizer. */
while (length > 0) {
if (src->bytes_in_buffer == 0) {
if (!(*src->fill_input_buffer) (cinfo))
return FALSE;
}
size_t available = (size_t)length < src->bytes_in_buffer ?
(size_t)length : src->bytes_in_buffer;
for (size_t i = 0; i < available; i++)
marker_sum += src->next_input_byte[i];
src->next_input_byte += available;
src->bytes_in_buffer -= available;
length -= (INT32)available;
}
return TRUE;
}
#define NUMTESTS 7
struct test {
J_COLOR_SPACE out_color_space;
boolean quantize_colors;
boolean two_pass_quantize;
J_DITHER_MODE dither_mode;
};
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
struct jpeg_decompress_struct cinfo;
struct fuzzer_error_mgr jerr;
JSAMPARRAY buffer = NULL;
int row_stride;
int numTests = 1;
struct test tests[NUMTESTS] = {
/*
Output Quantize 2-Pass Dither
Colorspace Colors Quant Mode
*/
{ JCS_RGB565, FALSE, FALSE, JDITHER_NONE },
{ JCS_RGB565, FALSE, FALSE, JDITHER_ORDERED },
{ JCS_UNKNOWN, TRUE, FALSE, JDITHER_NONE },
{ JCS_UNKNOWN, TRUE, FALSE, JDITHER_ORDERED },
{ JCS_UNKNOWN, TRUE, FALSE, JDITHER_FS },
{ JCS_UNKNOWN, TRUE, TRUE, JDITHER_NONE },
{ JCS_UNKNOWN, TRUE, TRUE, JDITHER_FS }
};
/* Reject too-small input. */
if (size < 2)
return 0;
cinfo.err = jpeg_std_error(&jerr.pub);
jerr.pub.error_exit = fuzzer_error_exit;
jerr.pub.emit_message = fuzzer_emit_message;
jpeg_create_decompress(&cinfo);
for (int ti = 0; ti < numTests; ti++) {
int64_t sum = 0;
marker_sum = 0;
if (setjmp(jerr.setjmp_buffer)) {
jpeg_abort_decompress(&cinfo);
continue;
}
jpeg_mem_src(&cinfo, data, (unsigned long)size);
for (int m = JPEG_APP0; m <= JPEG_APP0 + 15; m++) {
if (m != JPEG_APP0 + 3)
jpeg_save_markers(&cinfo, m, 256);
}
jpeg_set_marker_processor(&cinfo, JPEG_APP0 + 3, custom_marker_processor);
jpeg_read_header(&cinfo, TRUE);
/* Sanity check dimensions to avoid memory exhaustion. Casting width to
(uint64_t) prevents integer overflow if width * height > INT_MAX. */
if (cinfo.image_width < 1 || cinfo.image_height < 1 ||
(uint64_t)cinfo.image_width * cinfo.image_height > 1048576)
goto bailout;
cinfo.dct_method = JDCT_FLOAT;
cinfo.buffered_image = jpeg_has_multiple_scans(&cinfo);
if (((cinfo.jpeg_color_space == JCS_YCbCr ||
cinfo.jpeg_color_space == JCS_RGB) && cinfo.num_components == 3) ||
(cinfo.jpeg_color_space == JCS_GRAYSCALE &&
cinfo.num_components == 1)) {
cinfo.out_color_space = tests[ti].out_color_space;
if (cinfo.jpeg_color_space == JCS_GRAYSCALE) {
numTests = 5;
if (cinfo.out_color_space == JCS_UNKNOWN)
cinfo.out_color_space = JCS_GRAYSCALE;
} else {
numTests = 7;
if (cinfo.out_color_space == JCS_UNKNOWN)
cinfo.out_color_space = ti % 2 ? JCS_RGB : JCS_EXT_BGR;
}
cinfo.quantize_colors = tests[ti].quantize_colors;
cinfo.two_pass_quantize = tests[ti].two_pass_quantize;
cinfo.dither_mode = tests[ti].dither_mode;
}
if (!jpeg_start_decompress(&cinfo)) {
jpeg_abort_decompress(&cinfo);
continue;
}
row_stride = cinfo.output_width * cinfo.output_components;
buffer = (*cinfo.mem->alloc_sarray)
((j_common_ptr)&cinfo, JPOOL_IMAGE, row_stride, 1);
if (cinfo.buffered_image) {
/* Process all scans. */
while (!jpeg_input_complete(&cinfo) &&
cinfo.input_scan_number != cinfo.output_scan_number) {
int retval;
if (cinfo.input_scan_number > 100) {
jpeg_abort_decompress(&cinfo);
goto bailout;
}
/* Consume input data until we have a complete scan or reach the end
of input. */
do {
retval = jpeg_consume_input(&cinfo);
} while (retval != JPEG_SUSPENDED && retval != JPEG_REACHED_SOS &&
retval != JPEG_REACHED_EOI);
if (retval == JPEG_REACHED_EOI)
break;
/* Start outputting the current scan. */
if (!jpeg_start_output(&cinfo, cinfo.input_scan_number))
break;
while (cinfo.output_scanline < cinfo.output_height) {
if (!cinfo.two_pass_quantize &&
(cinfo.output_scanline == 0 || cinfo.output_scanline == 16)) {
JDIMENSION output_scanline = cinfo.output_scanline;
jpeg_skip_scanlines(&cinfo, 8);
if (cinfo.output_scanline == output_scanline)
break;
} else {
if (jpeg_read_scanlines(&cinfo, buffer, 1) != 1)
break;
/* Touch all of the output pixels in order to catch uninitialized
reads when using MemorySanitizer. */
for (int i = 0; i < row_stride; i++)
sum += buffer[0][i];
}
}
/* Finish this output pass. */
if (!jpeg_finish_output(&cinfo))
break;
}
} else {
while (cinfo.output_scanline < cinfo.output_height) {
if (!cinfo.two_pass_quantize &&
(cinfo.output_scanline == 0 || cinfo.output_scanline == 16))
jpeg_skip_scanlines(&cinfo, 8);
else {
jpeg_read_scanlines(&cinfo, buffer, 1);
for (int i = 0; i < row_stride; i++)
sum += buffer[0][i];
}
}
}
jpeg_finish_decompress(&cinfo);
/* Prevent the sums above from being optimized out. This test should never
be true, but the compiler doesn't know that. */
if (sum > (int64_t)255 * 1048576 * 4 ||
marker_sum > (int64_t)255 * 1048576)
goto bailout;
}
bailout:
jpeg_destroy_decompress(&cinfo);
return 0;
}
lib/libjpeg-turbo/fuzz/decompress_yuv.cc
Executable
+120
View File
@@ -0,0 +1,120 @@
/*
* Copyright (C) 2021-2026 D. R. Commander. All Rights Reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* - Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* - Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
* - Neither the name of the libjpeg-turbo Project nor the names of its
* contributors may be used to endorse or promote products derived from this
* software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS",
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include "../src/turbojpeg.h"
#include <stdlib.h>
#include <stdint.h>
#include <string.h>
#define NUMPF 4
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
tjhandle handle = NULL;
unsigned char *dstBuf = NULL, *yuvBuf = NULL;
int width = 0, height = 0, jpegSubsamp, pfi;
/* TJPF_RGB-TJPF_BGR share the same code paths, as do TJPF_RGBX-TJPF_XRGB and
TJPF_RGBA-TJPF_ARGB. Thus, the pixel formats below should be the minimum
necessary to achieve full coverage. */
enum TJPF pixelFormats[NUMPF] =
{ TJPF_BGR, TJPF_RGBA, TJPF_XRGB, TJPF_GRAY };
if ((handle = tj3Init(TJINIT_DECOMPRESS)) == NULL)
goto bailout;
/* We ignore the return value of tj3DecompressHeader(), because malformed
JPEG images that might expose issues in libjpeg-turbo might also have
header errors that cause tj3DecompressHeader() to fail. */
tj3DecompressHeader(handle, data, size);
width = tj3Get(handle, TJPARAM_JPEGWIDTH);
height = tj3Get(handle, TJPARAM_JPEGHEIGHT);
jpegSubsamp = tj3Get(handle, TJPARAM_SUBSAMP);
/* Ignore 0-pixel images and images larger than 1 Megapixel. Casting width
to (uint64_t) prevents integer overflow if width * height > INT_MAX. */
if (width < 1 || height < 1 || (uint64_t)width * height > 1048576)
goto bailout;
tj3Set(handle, TJPARAM_SCANLIMIT, 100);
for (pfi = 0; pfi < NUMPF; pfi++) {
int w = width, h = height;
int pf = pixelFormats[pfi], i, sum = 0;
/* Test non-default decompression options on the first iteration. */
if (!tj3Get(handle, TJPARAM_LOSSLESS)) {
tj3Set(handle, TJPARAM_BOTTOMUP, pfi == 0);
tj3Set(handle, TJPARAM_FASTUPSAMPLE, pfi == 0);
tj3Set(handle, TJPARAM_FASTDCT, pfi == 0);
/* Test IDCT scaling on the second and third iteration. */
if (pfi == 1 || pfi == 2) {
tjscalingfactor sf = { pfi == 1 ? 3 : 1, 4 };
tj3SetScalingFactor(handle, sf);
w = TJSCALED(width, sf);
h = TJSCALED(height, sf);
} else
tj3SetScalingFactor(handle, TJUNSCALED);
}
if ((dstBuf = (unsigned char *)tj3Alloc(w * h * tjPixelSize[pf])) == NULL)
goto bailout;
if ((yuvBuf =
(unsigned char *)tj3Alloc(tj3YUVBufSize(w, 1, h,
jpegSubsamp))) == NULL)
goto bailout;
if (tj3DecompressToYUV8(handle, data, size, yuvBuf, 1) == 0 &&
tj3DecodeYUV8(handle, yuvBuf, 1, dstBuf, w, 0, h, pf) == 0) {
/* Touch all of the output pixels in order to catch uninitialized reads
when using MemorySanitizer. */
for (i = 0; i < w * h * tjPixelSize[pf]; i++)
sum += dstBuf[i];
} else if (!strcmp(tj3GetErrorStr(handle),
"Progressive JPEG image has more than 100 scans"))
goto bailout;
tj3Free(dstBuf);
dstBuf = NULL;
tj3Free(yuvBuf);
yuvBuf = NULL;
/* Prevent the sum above from being optimized out. This test should never
be true, but the compiler doesn't know that. */
if (sum > 255 * 1048576 * tjPixelSize[pf])
goto bailout;
}
bailout:
tj3Free(dstBuf);
tj3Free(yuvBuf);
tj3Destroy(handle);
return 0;
}
lib/libjpeg-turbo/fuzz/jpeg.dict
Executable
+400
View File
@@ -0,0 +1,400 @@
# JPEG Dictionary for libFuzzer
# Contains JPEG markers, common signatures, and important byte patterns
# ==================================================
# JPEG markers (2-byte sequences starting with 0xFF)
# ==================================================
# Start Of Image/End Of Image (SOI/EOI)
soi="\xff\xd8"
eoi="\xff\xd9"
# Start Of Frame (SOF0-SOF15)
# Baseline DCT
sof0="\xff\xc0"
# Extended sequential DCT, Huffman coding
sof1="\xff\xc1"
# Progressive DCT, Huffman coding
sof2="\xff\xc2"
# Lossless, Huffman coding
sof3="\xff\xc3"
# Differential sequential DCT, Huffman coding
sof5="\xff\xc5"
# Differential progressive DCT, Huffman coding
sof6="\xff\xc6"
# Differential lossless, Huffman coding
sof7="\xff\xc7"
# Sequential DCT, arithmetic coding
sof9="\xff\xc9"
# Progressive DCT, arithmetic coding
sof10="\xff\xca"
# Lossless, arithmetic coding
sof11="\xff\xcb"
# Differential sequential DCT, arithmetic coding
sof13="\xff\xcd"
# Differential progressive DCT, arithmetic coding
sof14="\xff\xce"
# Differential lossless, arithmetic coding
sof15="\xff\xcf"
# Define Huffman Tables (DHT)
dht="\xff\xc4"
# Define Arithmetic Coding conditioning (DAC)
dac="\xff\xcc"
# Define Quantization Tables (DQT)
dqt="\xff\xdb"
# Define Restart Interval (DRI)
dri="\xff\xdd"
# Start Of Scan (SOS)
sos="\xff\xda"
# Restart (RST0-RST7)
rst0="\xff\xd0"
rst1="\xff\xd1"
rst2="\xff\xd2"
rst3="\xff\xd3"
rst4="\xff\xd4"
rst5="\xff\xd5"
rst6="\xff\xd6"
rst7="\xff\xd7"
# Application (APP0-APP15)
app0="\xff\xe0"
app1="\xff\xe1"
app2="\xff\xe2"
app3="\xff\xe3"
app4="\xff\xe4"
app5="\xff\xe5"
app6="\xff\xe6"
app7="\xff\xe7"
app8="\xff\xe8"
app9="\xff\xe9"
app10="\xff\xea"
app11="\xff\xeb"
app12="\xff\xec"
app13="\xff\xed"
app14="\xff\xee"
app15="\xff\xef"
# Comment (COM)
com="\xff\xfe"
# Define Number of Lines (DNL)
dnl="\xff\xdc"
# Expand reference components (EXP)
exp="\xff\xdf"
# JPEG extensions (JPG0-JPG13)
jpg0="\xff\xf0"
jpg1="\xff\xf1"
jpg2="\xff\xf2"
jpg3="\xff\xf3"
jpg4="\xff\xf4"
jpg5="\xff\xf5"
jpg6="\xff\xf6"
jpg7="\xff\xf7"
jpg8="\xff\xf8"
jpg9="\xff\xf9"
jpg10="\xff\xfa"
jpg11="\xff\xfb"
jpg12="\xff\xfc"
jpg13="\xff\xfd"
# Temporary (TEM)
tem="\xff\x01"
# Reserved (RES)
res_02="\xff\x02"
res_bf="\xff\xbf"
# Fill byte (byte stuffing)
fill="\xff\x00"
# ==============================
# Application segment signatures
# ==============================
# JFIF signature (in APP0)
jfif="JFIF\x00"
jfif_ver="\x01\x01"
jfif_ver2="\x01\x02"
# JFXX signature (in APP0)
jfxx="JFXX\x00"
# Exif signature (in APP1)
exif="Exif\x00\x00"
# XMP signature (in APP1)
xmp="http://ns.adobe.com/xap/1.0/\x00"
# ICC Profile signature (in APP2)
icc="ICC_PROFILE\x00"
# Adobe signature (in APP14)
adobe="Adobe\x00"
# Photoshop signature (in APP13)
photoshop="Photoshop 3.0\x008BIM"
# ============================
# TIFF/Exif byte order markers
# ============================
tiff_le="II\x2a\x00"
tiff_be="MM\x00\x2a"
# =================================
# Common length values (big-endian)
# =================================
len_2="\x00\x02"
len_4="\x00\x04"
len_8="\x00\x08"
len_16="\x00\x10"
len_17="\x00\x11"
len_32="\x00\x20"
len_64="\x00\x40"
len_128="\x00\x80"
len_256="\x01\x00"
len_512="\x02\x00"
len_1024="\x04\x00"
# ============================================
# Image dimensions (common values, big-endian)
# ============================================
dim_1="\x00\x01"
dim_8="\x00\x08"
dim_16="\x00\x10"
dim_64="\x00\x40"
dim_128="\x00\x80"
dim_256="\x01\x00"
dim_512="\x02\x00"
dim_1024="\x04\x00"
dim_2048="\x08\x00"
dim_4096="\x10\x00"
# ========================
# Component counts and IDs
# ========================
comp_1="\x01"
comp_2="\x02"
comp_3="\x03"
comp_4="\x04"
# Component IDs (Y, Cb, Cr)
comp_y="\x01"
comp_cb="\x02"
comp_cr="\x03"
comp_r="\x52"
comp_g="\x47"
comp_b="\x42"
# ===========================================
# Sampling factors (packed H:V into one byte)
# ===========================================
samp_11="\x11"
samp_21="\x21"
samp_12="\x12"
samp_22="\x22"
samp_41="\x41"
samp_14="\x14"
samp_44="\x44"
# ======================
# Quantization table IDs
# ======================
qt_0="\x00"
qt_1="\x01"
qt_2="\x02"
qt_3="\x03"
qt_16bit_0="\x10"
qt_16bit_1="\x11"
# =======================================
# Huffman table class and ID combinations
# =======================================
ht_dc_0="\x00"
ht_dc_1="\x01"
ht_dc_2="\x02"
ht_dc_3="\x03"
ht_ac_0="\x10"
ht_ac_1="\x11"
ht_ac_2="\x12"
ht_ac_3="\x13"
# =====================
# Data precision values
# =====================
prec_8="\x08"
prec_12="\x0c"
prec_16="\x10"
# =====================================
# Huffman code lengths (for DHT marker)
# =====================================
huff_0_codes="\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
huff_std_dc="\x00\x01\x05\x01\x01\x01\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00"
huff_std_ac="\x00\x02\x01\x03\x03\x02\x04\x03\x05\x05\x04\x04\x00\x00\x01\x7d"
# =======================
# Restart interval values
# =======================
ri_0="\x00\x00"
ri_1="\x00\x01"
ri_8="\x00\x08"
ri_16="\x00\x10"
ri_100="\x00\x64"
ri_256="\x01\x00"
# ==================
# Scan header values
# ==================
scan_start_0="\x00"
scan_start_1="\x01"
scan_end_0="\x00"
scan_end_63="\x3f"
scan_approx_0="\x00"
scan_approx_10="\x10"
scan_approx_01="\x01"
scan_approx_11="\x11"
scan_approx_21="\x21"
# ===================================
# Progressive scan approximation bits
# ===================================
ah_al_00="\x00"
ah_al_10="\x10"
ah_al_20="\x20"
ah_al_01="\x01"
ah_al_11="\x11"
ah_al_21="\x21"
ah_al_12="\x12"
# =========================
# Lossless predictor values
# =========================
pred_0="\x00"
pred_1="\x01"
pred_2="\x02"
pred_3="\x03"
pred_4="\x04"
pred_5="\x05"
pred_6="\x06"
pred_7="\x07"
# ==============================
# Common marker segment patterns
# ==============================
# Minimal DQT segment (64-byte table + header)
dqt_hdr="\xff\xdb\x00\x43\x00"
# Minimal DHT segment header
dht_hdr="\xff\xc4\x00\x1f\x00"
# Minimal SOF0 segment header (baseline)
sof0_hdr="\xff\xc0\x00\x0b\x08"
# Minimal SOS segment header
sos_hdr="\xff\xda\x00\x08\x01"
# Typical 3-component SOS
sos_3comp="\xff\xda\x00\x0c\x03\x01\x00\x02\x11\x03\x11\x00\x3f\x00"
# ================
# Edge case values
# ================
zero="\x00"
one="\x01"
max_byte="\xff"
mid="\x80"
val_7f="\x7f"
val_fe="\xfe"
# Large values (for dimension fuzzing)
large_dim="\xff\xff"
large_len="\xff\xfe"
# ===============================
# Entropy coding segment patterns
# ===============================
# Common DC coefficient patterns
dc_zero="\x00"
dc_small="\xf0"
# EOB (End Of Block) for AC
eob="\x00"
# ZRL (Zero Run Length) - 16 zeros
zrl="\xf0"
# ===========================
# JPEG file structure markers
# ===========================
# SOI + APP0 (JFIF header start)
soi_app0="\xff\xd8\xff\xe0"
# Minimal JFIF APP0 segment
jfif_app0="\xff\xe0\x00\x10JFIF\x00\x01\x01\x00\x00\x01\x00\x01\x00\x00"
# SOI + SOF0 (baseline start)
soi_sof0="\xff\xd8\xff\xc0"
# SOI + SOF2 (progressive start)
soi_sof2="\xff\xd8\xff\xc2"
# DQT + SOF sequence
dqt_sof="\xff\xdb\xff\xc0"
# SOF + DHT sequence
sof_dht="\xff\xc0\xff\xc4"
# DHT + SOS sequence
dht_sos="\xff\xc4\xff\xda"
# SOS + EOI (end of scan + End Of Image)
sos_eoi="\xff\xda\xff\xd9"
# ====================
# ICC profile patterns
# ====================
icc_sig="ICC_PROFILE\x00\x01\x01"
icc_multi_1="ICC_PROFILE\x00\x01\x02"
icc_multi_2="ICC_PROFILE\x00\x02\x02"
# ==========================
# Arithmetic coding patterns
# ==========================
arith_cond="\x00\x00"
arith_kx="\x00\x05"
# ====================================
# Color transform values (Adobe APP14)
# ====================================
adobe_transform_0="\x00"
adobe_transform_1="\x01"
adobe_transform_2="\x02"
lib/libjpeg-turbo/fuzz/transform.cc
Executable
+186
View File
@@ -0,0 +1,186 @@
/*
* Copyright (C) 2011, 2021-2026 D. R. Commander. All Rights Reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* - Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* - Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
* - Neither the name of the libjpeg-turbo Project nor the names of its
* contributors may be used to endorse or promote products derived from this
* software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS",
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include "../src/turbojpeg.h"
#include <stdlib.h>
#include <stdint.h>
#include <string.h>
static int dummyDCTFilter(short *coeffs, tjregion arrayRegion,
tjregion planeRegion, int componentIndex,
int transformIndex, tjtransform *transform)
{
int i;
for (i = 0; i < arrayRegion.w * arrayRegion.h; i++)
coeffs[i] = -coeffs[i];
return 0;
}
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
tjhandle handle = NULL;
unsigned char *dstBufs[1] = { NULL };
size_t dstSizes[1] = { 0 }, maxBufSize, i;
int width = 0, height = 0, jpegSubsamp;
tjtransform transforms[1];
if ((handle = tj3Init(TJINIT_TRANSFORM)) == NULL)
goto bailout;
/* We ignore the return value of tj3DecompressHeader(), because malformed
JPEG images that might expose issues in libjpeg-turbo might also have
header errors that cause tj3DecompressHeader() to fail. */
tj3DecompressHeader(handle, data, size);
width = tj3Get(handle, TJPARAM_JPEGWIDTH);
height = tj3Get(handle, TJPARAM_JPEGHEIGHT);
jpegSubsamp = tj3Get(handle, TJPARAM_SUBSAMP);
/* Let the transform options dictate the entropy coding algorithm. */
tj3Set(handle, TJPARAM_ARITHMETIC, 0);
tj3Set(handle, TJPARAM_PROGRESSIVE, 0);
tj3Set(handle, TJPARAM_OPTIMIZE, 0);
/* Ignore 0-pixel images and images larger than 1 Megapixel. Casting width
to (uint64_t) prevents integer overflow if width * height > INT_MAX. */
if (width < 1 || height < 1 || (uint64_t)width * height > 1048576)
goto bailout;
tj3Set(handle, TJPARAM_SCANLIMIT, 100);
if (jpegSubsamp < 0 || jpegSubsamp >= TJ_NUMSAMP)
jpegSubsamp = TJSAMP_444;
memset(&transforms[0], 0, sizeof(tjtransform));
transforms[0].op = TJXOP_NONE;
transforms[0].options = TJXOPT_PROGRESSIVE | TJXOPT_COPYNONE;
dstSizes[0] = maxBufSize = tj3TransformBufSize(handle, &transforms[0]);
if (dstSizes[0] == 0 ||
(dstBufs[0] = (unsigned char *)tj3Alloc(dstSizes[0])) == NULL)
goto bailout;
if (size >= 34)
tj3SetICCProfile(handle, (unsigned char *)&data[2], 32);
tj3Set(handle, TJPARAM_NOREALLOC, 1);
if (tj3Transform(handle, data, size, 1, dstBufs, dstSizes,
transforms) == 0) {
/* Touch all of the output data in order to catch uninitialized reads when
using MemorySanitizer. */
size_t sum = 0;
for (i = 0; i < dstSizes[0]; i++)
sum += dstBufs[0][i];
/* Prevent the sum above from being optimized out. This test should never
be true, but the compiler doesn't know that. */
if (sum > 255 * maxBufSize)
goto bailout;
} else if (!strcmp(tj3GetErrorStr(handle),
"Progressive JPEG image has more than 100 scans"))
goto bailout;
tj3Free(dstBufs[0]);
dstBufs[0] = NULL;
transforms[0].r.w = (height + 1) / 2;
transforms[0].r.h = (width + 1) / 2;
transforms[0].op = TJXOP_TRANSPOSE;
transforms[0].options = TJXOPT_GRAY | TJXOPT_CROP | TJXOPT_COPYNONE |
TJXOPT_OPTIMIZE;
dstSizes[0] = maxBufSize = tj3TransformBufSize(handle, &transforms[0]);
if (dstSizes[0] == 0 ||
(dstBufs[0] = (unsigned char *)tj3Alloc(dstSizes[0])) == NULL)
goto bailout;
if (tj3Transform(handle, data, size, 1, dstBufs, dstSizes,
transforms) == 0) {
size_t sum = 0;
for (i = 0; i < dstSizes[0]; i++)
sum += dstBufs[0][i];
if (sum > 255 * maxBufSize)
goto bailout;
} else if (!strcmp(tj3GetErrorStr(handle),
"Progressive JPEG image has more than 100 scans"))
goto bailout;
tj3Free(dstBufs[0]);
dstBufs[0] = NULL;
transforms[0].op = TJXOP_ROT90;
transforms[0].options = TJXOPT_TRIM | TJXOPT_ARITHMETIC;
dstSizes[0] = maxBufSize = tj3TransformBufSize(handle, &transforms[0]);
if (dstSizes[0] == 0 ||
(dstBufs[0] = (unsigned char *)tj3Alloc(dstSizes[0])) == NULL)
goto bailout;
if (tj3Transform(handle, data, size, 1, dstBufs, dstSizes,
transforms) == 0) {
size_t sum = 0;
for (i = 0; i < dstSizes[0]; i++)
sum += dstBufs[0][i];
if (sum > 255 * maxBufSize)
goto bailout;
} else if (!strcmp(tj3GetErrorStr(handle),
"Progressive JPEG image has more than 100 scans"))
goto bailout;
tj3Free(dstBufs[0]);
dstBufs[0] = NULL;
transforms[0].op = TJXOP_NONE;
transforms[0].options = TJXOPT_PROGRESSIVE;
transforms[0].customFilter = dummyDCTFilter;
dstSizes[0] = 0;
tj3Set(handle, TJPARAM_NOREALLOC, 0);
if (tj3Transform(handle, data, size, 1, dstBufs, dstSizes,
transforms) == 0) {
size_t sum = 0;
for (i = 0; i < dstSizes[0]; i++)
sum += dstBufs[0][i];
if (sum > 255 * maxBufSize)
goto bailout;
} else if (!strcmp(tj3GetErrorStr(handle),
"Progressive JPEG image has more than 100 scans"))
goto bailout;
bailout:
tj3Free(dstBufs[0]);
tj3Destroy(handle);
return 0;
}
lib/libjpeg-turbo/jconfig.h
Executable
+60
View File
@@ -0,0 +1,60 @@
/* Version ID for the JPEG library.
* Might be useful for tests like "#if JPEG_LIB_VERSION >= 60".
*/
#define JPEG_LIB_VERSION 62
/* libjpeg-turbo version */
#define LIBJPEG_TURBO_VERSION 3.1.4
/* libjpeg-turbo version in integer form */
#define LIBJPEG_TURBO_VERSION_NUMBER 3001004
/* Support arithmetic encoding when using 8-bit samples */
#define C_ARITH_CODING_SUPPORTED 1
/* Support arithmetic decoding when using 8-bit samples */
#define D_ARITH_CODING_SUPPORTED 1
/* Support in-memory source/destination managers */
#define MEM_SRCDST_SUPPORTED 1
/* Use accelerated SIMD routines when using 8-bit samples */
#define WITH_SIMD 1
/* This version of libjpeg-turbo supports run-time selection of data precision,
* so BITS_IN_JSAMPLE is no longer used to specify the data precision at build
* time. However, some downstream software expects the macro to be defined.
* Since 12-bit data precision is an opt-in feature that requires explicitly
* calling 12-bit-specific libjpeg API functions and using 12-bit-specific data
* types, the unmodified portion of the libjpeg API still behaves as if it were
* built for 8-bit precision, and JSAMPLE is still literally an 8-bit data
* type. Thus, it is correct to define BITS_IN_JSAMPLE to 8 here.
*/
#ifndef BITS_IN_JSAMPLE
#define BITS_IN_JSAMPLE 8
#endif
#ifdef _WIN32
#undef RIGHT_SHIFT_IS_UNSIGNED
/* Define "boolean" as unsigned char, not int, per Windows custom */
#ifndef __RPCNDR_H__ /* don't conflict if rpcndr.h already read */
typedef unsigned char boolean;
#endif
#define HAVE_BOOLEAN /* prevent jmorecfg.h from redefining it */
/* Define "INT32" as int, not long, per Windows custom */
#if !(defined(_BASETSD_H_) || defined(_BASETSD_H)) /* don't conflict if basetsd.h already read */
typedef short INT16;
typedef signed int INT32;
#endif
#define XMD_H /* prevent jmorecfg.h from redefining it */
#else
/* Define if your (broken) compiler shifts signed values as if they were
unsigned. */
/* #undef RIGHT_SHIFT_IS_UNSIGNED */
#endif
lib/libjpeg-turbo/jconfigint.h
Executable
+76
View File
@@ -0,0 +1,76 @@
/* libjpeg-turbo build number */
#define BUILD "20260221"
/* How to hide global symbols. */
#define HIDDEN
/* Compiler's inline keyword */
#undef inline
/* How to obtain function inlining. */
#define INLINE __inline__ __attribute__((always_inline))
/* How to obtain thread-local storage */
#define THREAD_LOCAL __thread
/* Define to the full name of this package. */
#define PACKAGE_NAME "libjpeg-turbo"
/* Version number of package */
#define VERSION "3.1.4"
/* The size of `size_t', as computed by sizeof. */
#define SIZEOF_SIZE_T 8
/* Define if your compiler has __builtin_ctzl() and sizeof(unsigned long) == sizeof(size_t). */
/* #undef HAVE_BUILTIN_CTZL */
/* Define to 1 if you have the <intrin.h> header file. */
/* #undef HAVE_INTRIN_H */
#if defined(_MSC_VER) && defined(HAVE_INTRIN_H)
#if (SIZEOF_SIZE_T == 8)
#define HAVE_BITSCANFORWARD64
#elif (SIZEOF_SIZE_T == 4)
#define HAVE_BITSCANFORWARD
#endif
#endif
#if defined(__has_attribute)
#if __has_attribute(fallthrough)
#define FALLTHROUGH __attribute__((fallthrough));
#else
#define FALLTHROUGH
#endif
#else
#define FALLTHROUGH
#endif
/*
* Define BITS_IN_JSAMPLE as either
* 8 for 8-bit sample values (the usual setting)
* 12 for 12-bit sample values
* Only 8 and 12 are legal data precisions for lossy JPEG according to the
* JPEG standard, and the IJG code does not support anything else!
*/
#ifndef BITS_IN_JSAMPLE
#define BITS_IN_JSAMPLE 8 /* use 8 or 12 */
#endif
#undef C_ARITH_CODING_SUPPORTED
#undef D_ARITH_CODING_SUPPORTED
#undef WITH_SIMD
#if BITS_IN_JSAMPLE == 8
/* Support arithmetic encoding */
#define C_ARITH_CODING_SUPPORTED 1
/* Support arithmetic decoding */
#define D_ARITH_CODING_SUPPORTED 1
/* Use accelerated SIMD routines. */
#define WITH_SIMD 1
#endif
lib/libjpeg-turbo/jversion.h
Executable
+56
View File
@@ -0,0 +1,56 @@
/*
* jversion.h
*
* This file was part of the Independent JPEG Group's software:
* Copyright (C) 1991-2020, Thomas G. Lane, Guido Vollbeding.
* libjpeg-turbo Modifications:
* Copyright (C) 2010, 2012-2026, D. R. Commander.
* For conditions of distribution and use, see the accompanying README.ijg
* file.
*
* This file contains software version identification.
*/
#if JPEG_LIB_VERSION >= 80
#define JVERSION "8d 15-Jan-2012"
#elif JPEG_LIB_VERSION >= 70
#define JVERSION "7 27-Jun-2009"
#else
#define JVERSION "6b 27-Mar-1998"
#endif
/*
* NOTE: It is our convention to place the authors in the following order:
* - libjpeg-turbo authors (2009-) in descending order of the date of their
* most recent contribution to the project, then in ascending order of the
* date of their first contribution to the project, then in alphabetical
* order
* - Upstream authors in descending order of the date of the first inclusion of
* their code
*/
#define JCOPYRIGHT1 \
"Copyright (C) 2009-2026 D. R. Commander\n" \
"Copyright (C) 2015-2016, 2018, 2022 Matthieu Darbois\n" \
"Copyright (C) 2019-2021 Arm Limited\n" \
"Copyright (C) 2015, 2020 Google, Inc.\n" \
"Copyright (C) 2011, 2014, 2016 Siarhei Siamashka\n" \
"Copyright (C) 2015 Intel Corporation\n"
#define JCOPYRIGHT2 \
"Copyright (C) 2013-2014 Linaro Limited\n" \
"Copyright (C) 2013-2014 MIPS Technologies, Inc.\n" \
"Copyright (C) 2009, 2012 Pierre Ossman for Cendio AB\n" \
"Copyright (C) 2009-2011 Nokia Corporation and/or its subsidiary(-ies)\n" \
"Copyright (C) 1999-2006 MIYASAKA Masaru\n" \
"Copyright (C) 1999 Ken Murchison\n" \
"Copyright (C) 1991-2020 Thomas G. Lane, Guido Vollbeding\n"
#define JCOPYRIGHT_SHORT \
"Copyright (C) 1991-2026 The libjpeg-turbo Project and many others"
lib/libjpeg-turbo/simd/CMakeLists.txt
Executable
+574
View File
@@ -0,0 +1,574 @@
macro(simd_fail message)
if(REQUIRE_SIMD)
message(FATAL_ERROR "${message}.")
else()
message(WARNING "${message}. Performance will suffer.")
set(WITH_SIMD 0 PARENT_SCOPE)
endif()
endmacro()
###############################################################################
# x86[-64] (NASM)
###############################################################################
if(CPU_TYPE STREQUAL "x86_64" OR CPU_TYPE STREQUAL "i386")
set(CMAKE_ASM_NASM_FLAGS_DEBUG_INIT "-g")
set(CMAKE_ASM_NASM_FLAGS_RELWITHDEBINFO_INIT "-g")
# Allow the location of the NASM executable to be specified using the ASM_NASM
# environment variable. This should happen automatically, but unfortunately
# enable_language(ASM_NASM) doesn't parse the ASM_NASM environment variable
# until after CMAKE_ASM_NASM_COMPILER has been populated with the results of
# searching for NASM or Yasm in the PATH.
if(NOT DEFINED CMAKE_ASM_NASM_COMPILER AND DEFINED ENV{ASM_NASM})
set(CMAKE_ASM_NASM_COMPILER $ENV{ASM_NASM})
endif()
if(CPU_TYPE STREQUAL "x86_64")
if(CYGWIN)
set(CMAKE_ASM_NASM_OBJECT_FORMAT win64)
endif()
if(CMAKE_C_COMPILER_ABI MATCHES "ELF X32")
set(CMAKE_ASM_NASM_OBJECT_FORMAT elfx32)
endif()
elseif(CPU_TYPE STREQUAL "i386")
if(BORLAND)
set(CMAKE_ASM_NASM_OBJECT_FORMAT obj)
elseif(CYGWIN)
set(CMAKE_ASM_NASM_OBJECT_FORMAT win32)
endif()
endif()
if(NOT REQUIRE_SIMD)
include(CheckLanguage)
check_language(ASM_NASM)
if(NOT CMAKE_ASM_NASM_COMPILER)
simd_fail("SIMD extensions disabled: could not find NASM compiler")
return()
endif()
endif()
enable_language(ASM_NASM)
message(STATUS "CMAKE_ASM_NASM_COMPILER = ${CMAKE_ASM_NASM_COMPILER}")
if(CMAKE_ASM_NASM_OBJECT_FORMAT MATCHES "^macho")
set(CMAKE_ASM_NASM_FLAGS "${CMAKE_ASM_NASM_FLAGS} -DMACHO")
elseif(CMAKE_ASM_NASM_OBJECT_FORMAT MATCHES "^elf")
set(CMAKE_ASM_NASM_FLAGS "${CMAKE_ASM_NASM_FLAGS} -DELF")
set(CMAKE_ASM_NASM_DEBUG_FORMAT "dwarf2")
endif()
if(CPU_TYPE STREQUAL "x86_64")
if(WIN32 OR CYGWIN)
set(CMAKE_ASM_NASM_FLAGS "${CMAKE_ASM_NASM_FLAGS} -DWIN64")
endif()
set(CMAKE_ASM_NASM_FLAGS "${CMAKE_ASM_NASM_FLAGS} -D__x86_64__")
elseif(CPU_TYPE STREQUAL "i386")
if(BORLAND)
set(CMAKE_ASM_NASM_FLAGS "${CMAKE_ASM_NASM_FLAGS} -DOBJ32")
elseif(WIN32 OR CYGWIN)
set(CMAKE_ASM_NASM_FLAGS "${CMAKE_ASM_NASM_FLAGS} -DWIN32")
endif()
endif()
message(STATUS "CMAKE_ASM_NASM_OBJECT_FORMAT = ${CMAKE_ASM_NASM_OBJECT_FORMAT}")
if(NOT CMAKE_ASM_NASM_OBJECT_FORMAT)
simd_fail("SIMD extensions disabled: could not determine NASM object format")
return()
endif()
get_filename_component(CMAKE_ASM_NASM_COMPILER_TYPE
"${CMAKE_ASM_NASM_COMPILER}" NAME_WE)
if(CMAKE_ASM_NASM_COMPILER_TYPE MATCHES "yasm")
foreach(var CMAKE_ASM_NASM_FLAGS_DEBUG CMAKE_ASM_NASM_FLAGS_RELWITHDEBINFO)
if(${var} STREQUAL "-g")
if(CMAKE_ASM_NASM_DEBUG_FORMAT)
set_property(CACHE ${var} PROPERTY VALUE "-g ${CMAKE_ASM_NASM_DEBUG_FORMAT}")
else()
set_property(CACHE ${var} PROPERTY VALUE "")
endif()
endif()
endforeach()
endif()
if(NOT WIN32 AND (CMAKE_POSITION_INDEPENDENT_CODE OR ENABLE_SHARED))
set(CMAKE_ASM_NASM_FLAGS "${CMAKE_ASM_NASM_FLAGS} -DPIC")
endif()
if(CPU_TYPE STREQUAL "x86_64" AND CMAKE_ASM_NASM_OBJECT_FORMAT MATCHES "^elf")
check_c_source_compiles("
#if (__CET__ & 3) == 0
#error \"CET not enabled\"
#endif
int main(void) { return 0; }" HAVE_CET)
if(HAVE_CET)
set(CMAKE_ASM_NASM_FLAGS "${CMAKE_ASM_NASM_FLAGS} -D__CET__")
endif()
endif()
string(TOUPPER ${CMAKE_BUILD_TYPE} CMAKE_BUILD_TYPE_UC)
set(EFFECTIVE_ASM_NASM_FLAGS "${CMAKE_ASM_NASM_FLAGS} ${CMAKE_ASM_NASM_FLAGS_${CMAKE_BUILD_TYPE_UC}}")
message(STATUS "CMAKE_ASM_NASM_FLAGS = ${EFFECTIVE_ASM_NASM_FLAGS}")
set(CMAKE_ASM_NASM_FLAGS "${CMAKE_ASM_NASM_FLAGS} -I\"${CMAKE_CURRENT_SOURCE_DIR}/nasm/\" -I\"${CMAKE_CURRENT_SOURCE_DIR}/${CPU_TYPE}/\"")
set(GREP grep)
if(CMAKE_SYSTEM_NAME STREQUAL "SunOS")
set(GREP ggrep)
endif()
add_custom_target(jsimdcfg COMMAND
${CMAKE_C_COMPILER} -E -I${CMAKE_BINARY_DIR} -I${CMAKE_CURRENT_BINARY_DIR}
-I${CMAKE_CURRENT_SOURCE_DIR}
${CMAKE_CURRENT_SOURCE_DIR}/nasm/jsimdcfg.inc.h |
${GREP} -E '^[\;%]|^\ %' | sed 's%_cpp_protection_%%' |
sed 's@% define@%define@g' >${CMAKE_CURRENT_SOURCE_DIR}/nasm/jsimdcfg.inc)
if(CPU_TYPE STREQUAL "x86_64")
set(SIMD_SOURCES x86_64/jsimdcpu.asm x86_64/jfdctflt-sse.asm
x86_64/jccolor-sse2.asm x86_64/jcgray-sse2.asm x86_64/jchuff-sse2.asm
x86_64/jcphuff-sse2.asm x86_64/jcsample-sse2.asm x86_64/jdcolor-sse2.asm
x86_64/jdmerge-sse2.asm x86_64/jdsample-sse2.asm x86_64/jfdctfst-sse2.asm
x86_64/jfdctint-sse2.asm x86_64/jidctflt-sse2.asm x86_64/jidctfst-sse2.asm
x86_64/jidctint-sse2.asm x86_64/jidctred-sse2.asm x86_64/jquantf-sse2.asm
x86_64/jquanti-sse2.asm
x86_64/jccolor-avx2.asm x86_64/jcgray-avx2.asm x86_64/jcsample-avx2.asm
x86_64/jdcolor-avx2.asm x86_64/jdmerge-avx2.asm x86_64/jdsample-avx2.asm
x86_64/jfdctint-avx2.asm x86_64/jidctint-avx2.asm x86_64/jquanti-avx2.asm)
else()
set(SIMD_SOURCES i386/jsimdcpu.asm i386/jfdctflt-3dn.asm
i386/jidctflt-3dn.asm i386/jquant-3dn.asm
i386/jccolor-mmx.asm i386/jcgray-mmx.asm i386/jcsample-mmx.asm
i386/jdcolor-mmx.asm i386/jdmerge-mmx.asm i386/jdsample-mmx.asm
i386/jfdctfst-mmx.asm i386/jfdctint-mmx.asm i386/jidctfst-mmx.asm
i386/jidctint-mmx.asm i386/jidctred-mmx.asm i386/jquant-mmx.asm
i386/jfdctflt-sse.asm i386/jidctflt-sse.asm i386/jquant-sse.asm
i386/jccolor-sse2.asm i386/jcgray-sse2.asm i386/jchuff-sse2.asm
i386/jcphuff-sse2.asm i386/jcsample-sse2.asm i386/jdcolor-sse2.asm
i386/jdmerge-sse2.asm i386/jdsample-sse2.asm i386/jfdctfst-sse2.asm
i386/jfdctint-sse2.asm i386/jidctflt-sse2.asm i386/jidctfst-sse2.asm
i386/jidctint-sse2.asm i386/jidctred-sse2.asm i386/jquantf-sse2.asm
i386/jquanti-sse2.asm
i386/jccolor-avx2.asm i386/jcgray-avx2.asm i386/jcsample-avx2.asm
i386/jdcolor-avx2.asm i386/jdmerge-avx2.asm i386/jdsample-avx2.asm
i386/jfdctint-avx2.asm i386/jidctint-avx2.asm i386/jquanti-avx2.asm)
endif()
if(MSVC_IDE)
set(OBJDIR "${CMAKE_CURRENT_BINARY_DIR}/${CMAKE_CFG_INTDIR}")
string(REGEX REPLACE " " ";" CMAKE_ASM_NASM_FLAGS "${CMAKE_ASM_NASM_FLAGS}")
elseif(XCODE)
set(OBJDIR "${CMAKE_CURRENT_BINARY_DIR}")
string(REGEX REPLACE " " ";" CMAKE_ASM_NASM_FLAGS "${CMAKE_ASM_NASM_FLAGS}")
endif()
file(GLOB INC_FILES nasm/*.inc)
foreach(file ${SIMD_SOURCES})
set(OBJECT_DEPENDS "")
if(${file} MATCHES jccolor)
string(REGEX REPLACE "jccolor" "jccolext" DEPFILE ${file})
set(OBJECT_DEPENDS ${OBJECT_DEPENDS}
${CMAKE_CURRENT_SOURCE_DIR}/${DEPFILE})
endif()
if(${file} MATCHES jcgray)
string(REGEX REPLACE "jcgray" "jcgryext" DEPFILE ${file})
set(OBJECT_DEPENDS ${OBJECT_DEPENDS}
${CMAKE_CURRENT_SOURCE_DIR}/${DEPFILE})
endif()
if(${file} MATCHES jdcolor)
string(REGEX REPLACE "jdcolor" "jdcolext" DEPFILE ${file})
set(OBJECT_DEPENDS ${OBJECT_DEPENDS}
${CMAKE_CURRENT_SOURCE_DIR}/${DEPFILE})
endif()
if(${file} MATCHES jdmerge)
string(REGEX REPLACE "jdmerge" "jdmrgext" DEPFILE ${file})
set(OBJECT_DEPENDS ${OBJECT_DEPENDS}
${CMAKE_CURRENT_SOURCE_DIR}/${DEPFILE})
endif()
set(OBJECT_DEPENDS ${OBJECT_DEPENDS} ${INC_FILES})
if(MSVC_IDE OR XCODE)
# The CMake Visual Studio generators do not work properly with the ASM_NASM
# language, so we have to go rogue here and use a custom command like we
# did in prior versions of libjpeg-turbo. (This is why we can't have nice
# things.)
string(REGEX REPLACE "${CPU_TYPE}/" "" filename ${file})
set(SIMD_OBJ ${OBJDIR}/${filename}${CMAKE_C_OUTPUT_EXTENSION})
add_custom_command(OUTPUT ${SIMD_OBJ} DEPENDS ${file} ${OBJECT_DEPENDS}
COMMAND ${CMAKE_ASM_NASM_COMPILER} -f${CMAKE_ASM_NASM_OBJECT_FORMAT}
${CMAKE_ASM_NASM_FLAGS} ${CMAKE_CURRENT_SOURCE_DIR}/${file}
-o${SIMD_OBJ})
set(SIMD_OBJS ${SIMD_OBJS} ${SIMD_OBJ})
else()
set_source_files_properties(${file} PROPERTIES OBJECT_DEPENDS
"${OBJECT_DEPENDS}")
endif()
endforeach()
if(MSVC_IDE OR XCODE)
set(SIMD_OBJS ${SIMD_OBJS} PARENT_SCOPE)
add_library(simd OBJECT ${CPU_TYPE}/jsimd.c)
add_custom_target(simd-objs DEPENDS ${SIMD_OBJS})
add_dependencies(simd simd-objs)
else()
add_library(simd OBJECT ${SIMD_SOURCES} ${CPU_TYPE}/jsimd.c)
endif()
if(NOT WIN32 AND (CMAKE_POSITION_INDEPENDENT_CODE OR ENABLE_SHARED))
set_target_properties(simd PROPERTIES POSITION_INDEPENDENT_CODE 1)
endif()
###############################################################################
# Arm (Intrinsics or GAS)
###############################################################################
elseif(CPU_TYPE STREQUAL "arm64" OR CPU_TYPE STREQUAL "arm")
# If Neon instructions are not explicitly enabled at compile time (e.g. using
# -mfpu=neon) with an AArch32 Linux or Android build, then the AArch32 SIMD
# dispatcher will parse /proc/cpuinfo to determine whether the Neon SIMD
# extensions can be enabled at run time. In order to support all AArch32 CPUs
# using the same code base, i.e. to support run-time FPU and Neon
# auto-detection, it is necessary to compile the scalar C source code using
# -mfloat-abi=soft (which is usually the default) but compile the intrinsics
# implementation of the Neon SIMD extensions using -mfloat-abi=softfp. The
# following test determines whether -mfloat-abi=softfp should be explicitly
# added to the compile flags for the intrinsics implementation of the Neon SIMD
# extensions.
if(BITS EQUAL 32)
check_c_source_compiles("
#if defined(__ARM_NEON__) || (!defined(__linux__) && !defined(ANDROID) && !defined(__ANDROID__))
#error \"Neon run-time auto-detection will not be used\"
#endif
#if __ARM_PCS_VFP == 1
#error \"float ABI = hard\"
#endif
#if __SOFTFP__ != 1
#error \"float ABI = softfp\"
#endif
int main(void) { return 0; }" NEED_SOFTFP_FOR_INTRINSICS)
if(NEED_SOFTFP_FOR_INTRINSICS)
set(SOFTFP_FLAG -mfloat-abi=softfp)
endif()
endif()
if(BITS EQUAL 32)
set(CMAKE_REQUIRED_FLAGS "-mfpu=neon ${SOFTFP_FLAG}")
check_c_source_compiles("
#include <arm_neon.h>
int main(int argc, char **argv) {
uint16x8_t input = vdupq_n_u16((uint16_t)argc);
uint8x8_t output = vmovn_u16(input);
return (int)output[0];
}" HAVE_NEON)
if(NOT HAVE_NEON)
simd_fail("SIMD extensions not available for this architecture")
return()
endif()
endif()
check_c_source_compiles("
#include <arm_neon.h>
int main(int argc, char **argv) {
int16_t input[12];
int16x4x3_t output;
int i;
for (i = 0; i < 12; i++) input[i] = (int16_t)argc;
output = vld1_s16_x3(input);
vst3_s16(input, output);
return (int)input[0];
}" HAVE_VLD1_S16_X3)
check_c_source_compiles("
#include <arm_neon.h>
int main(int argc, char **argv) {
uint16_t input[8];
uint16x4x2_t output;
int i;
for (i = 0; i < 8; i++) input[i] = (uint16_t)argc;
output = vld1_u16_x2(input);
vst2_u16(input, output);
return (int)input[0];
}" HAVE_VLD1_U16_X2)
check_c_source_compiles("
#include <arm_neon.h>
int main(int argc, char **argv) {
uint8_t input[64];
uint8x16x4_t output;
int i;
for (i = 0; i < 64; i++) input[i] = (uint8_t)argc;
output = vld1q_u8_x4(input);
vst4q_u8(input, output);
return (int)input[0];
}" HAVE_VLD1Q_U8_X4)
if(BITS EQUAL 32)
unset(CMAKE_REQUIRED_FLAGS)
endif()
configure_file(arm/neon-compat.h.in arm/neon-compat.h @ONLY)
include_directories(${CMAKE_CURRENT_BINARY_DIR}/arm)
# GCC 11 and earlier and some older versions of Clang do not have a full or
# optimal set of Neon intrinsics, so for performance reasons, when using those
# compilers, we default to using the older GAS implementation of the Neon SIMD
# extensions for certain algorithms. The presence or absence of the three
# intrinsics we tested above is a reasonable proxy for this, except with GCC 10
# and 11.
if((HAVE_VLD1_S16_X3 AND HAVE_VLD1_U16_X2 AND HAVE_VLD1Q_U8_X4 AND
(NOT CMAKE_COMPILER_IS_GNUCC OR
CMAKE_C_COMPILER_VERSION VERSION_EQUAL 12.0.0 OR
CMAKE_C_COMPILER_VERSION VERSION_GREATER 12.0.0)))
set(DEFAULT_NEON_INTRINSICS 1)
else()
set(DEFAULT_NEON_INTRINSICS 0)
endif()
option(NEON_INTRINSICS
"Because GCC (as of this writing) and some older versions of Clang do not have a full or optimal set of Neon intrinsics, for performance reasons, the default when building libjpeg-turbo with those compilers is to continue using the older GAS implementation of the Neon SIMD extensions for certain algorithms. Setting this option forces the full Neon intrinsics implementation to be used with all compilers. Unsetting this option forces the hybrid GAS/intrinsics implementation to be used with all compilers."
${DEFAULT_NEON_INTRINSICS})
if(NOT NEON_INTRINSICS)
enable_language(ASM)
set(CMAKE_ASM_FLAGS "${CMAKE_C_FLAGS} ${CMAKE_ASM_FLAGS}")
# Test whether gas-preprocessor.pl would be needed to build the GAS
# implementation of the Neon SIMD extensions. If so, then automatically
# enable the full Neon intrinsics implementation.
if(CPU_TYPE STREQUAL "arm")
file(WRITE ${CMAKE_CURRENT_BINARY_DIR}/gastest.S "
.text
.fpu neon
.arch armv7a
.object_arch armv4
.arm
pld [r0]
vmovn.u16 d0, q0")
else()
file(WRITE ${CMAKE_CURRENT_BINARY_DIR}/gastest.S "
.text
MYVAR .req x0
movi v0.16b, #100
mov MYVAR, #100
.unreq MYVAR")
endif()
separate_arguments(CMAKE_ASM_FLAGS_SEP UNIX_COMMAND "${CMAKE_ASM_FLAGS}")
execute_process(COMMAND ${CMAKE_ASM_COMPILER} ${CMAKE_ASM_FLAGS_SEP}
-x assembler-with-cpp -c ${CMAKE_CURRENT_BINARY_DIR}/gastest.S
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} RESULT_VARIABLE RESULT
OUTPUT_VARIABLE OUTPUT ERROR_VARIABLE ERROR)
if(NOT RESULT EQUAL 0)
message(WARNING "GAS appears to be broken. Using the full Neon SIMD intrinsics implementation.")
set(NEON_INTRINSICS 1 CACHE INTERNAL "" FORCE)
endif()
endif()
boolean_number(NEON_INTRINSICS PARENT_SCOPE)
if(NEON_INTRINSICS)
add_definitions(-DNEON_INTRINSICS)
message(STATUS "Use full Neon SIMD intrinsics implementation (NEON_INTRINSICS = ${NEON_INTRINSICS})")
else()
message(STATUS "Use partial Neon SIMD intrinsics implementation (NEON_INTRINSICS = ${NEON_INTRINSICS})")
endif()
set(SIMD_SOURCES arm/jcgray-neon.c arm/jcphuff-neon.c arm/jcsample-neon.c
arm/jdmerge-neon.c arm/jdsample-neon.c arm/jfdctfst-neon.c
arm/jidctred-neon.c arm/jquanti-neon.c)
if(NEON_INTRINSICS)
set(SIMD_SOURCES ${SIMD_SOURCES} arm/jccolor-neon.c arm/jidctint-neon.c)
endif()
if(NEON_INTRINSICS OR BITS EQUAL 64)
set(SIMD_SOURCES ${SIMD_SOURCES} arm/jidctfst-neon.c)
endif()
if(NEON_INTRINSICS OR BITS EQUAL 32)
set(SIMD_SOURCES ${SIMD_SOURCES} arm/aarch${BITS}/jchuff-neon.c
arm/jdcolor-neon.c arm/jfdctint-neon.c)
endif()
if(BITS EQUAL 32)
set_source_files_properties(${SIMD_SOURCES} COMPILE_FLAGS "-mfpu=neon ${SOFTFP_FLAG}")
endif()
if(NOT NEON_INTRINSICS)
string(TOUPPER ${CMAKE_BUILD_TYPE} CMAKE_BUILD_TYPE_UC)
set(EFFECTIVE_ASM_FLAGS "${CMAKE_ASM_FLAGS} ${CMAKE_ASM_FLAGS_${CMAKE_BUILD_TYPE_UC}}")
message(STATUS "CMAKE_ASM_FLAGS = ${EFFECTIVE_ASM_FLAGS}")
set(SIMD_SOURCES ${SIMD_SOURCES} arm/aarch${BITS}/jsimd_neon.S)
endif()
if(UNIX AND BITS EQUAL 32)
include(CheckSymbolExists)
check_symbol_exists(getauxval sys/auxv.h HAVE_GETAUXVAL)
if(HAVE_GETAUXVAL)
set_source_files_properties(arm/aarch${BITS}/jsimd.c PROPERTIES
COMPILE_DEFINITIONS HAVE_GETAUXVAL)
endif()
check_symbol_exists(elf_aux_info sys/auxv.h HAVE_ELF_AUX_INFO)
if(HAVE_ELF_AUX_INFO)
set_source_files_properties(arm/aarch${BITS}/jsimd.c PROPERTIES
COMPILE_DEFINITIONS HAVE_ELF_AUX_INFO)
endif()
endif()
add_library(simd OBJECT ${SIMD_SOURCES} arm/aarch${BITS}/jsimd.c)
if(CMAKE_POSITION_INDEPENDENT_CODE OR ENABLE_SHARED)
set_target_properties(simd PROPERTIES POSITION_INDEPENDENT_CODE 1)
endif()
###############################################################################
# MIPS (GAS)
###############################################################################
elseif(CPU_TYPE STREQUAL "mips" OR CPU_TYPE STREQUAL "mipsel")
enable_language(ASM)
string(TOUPPER ${CMAKE_BUILD_TYPE} CMAKE_BUILD_TYPE_UC)
set(EFFECTIVE_ASM_FLAGS "${CMAKE_ASM_FLAGS} ${CMAKE_ASM_FLAGS_${CMAKE_BUILD_TYPE_UC}}")
message(STATUS "CMAKE_ASM_FLAGS = ${EFFECTIVE_ASM_FLAGS}")
set(CMAKE_REQUIRED_FLAGS -mdspr2)
check_c_source_compiles("
#if !(defined(__mips__) && __mips_isa_rev >= 2)
#error MIPS DSPr2 is currently only available on MIPS32r2 platforms.
#endif
int main(void) {
int c = 0, a = 0, b = 0;
__asm__ __volatile__ (
\"precr.qb.ph %[c], %[a], %[b]\"
: [c] \"=r\" (c)
: [a] \"r\" (a), [b] \"r\" (b)
);
return c;
}" HAVE_DSPR2)
unset(CMAKE_REQUIRED_FLAGS)
if(NOT HAVE_DSPR2)
simd_fail("SIMD extensions not available for this CPU")
return()
endif()
add_library(simd OBJECT mips/jsimd_dspr2.S mips/jsimd.c)
if(CMAKE_POSITION_INDEPENDENT_CODE OR ENABLE_SHARED)
set_target_properties(simd PROPERTIES POSITION_INDEPENDENT_CODE 1)
endif()
###############################################################################
# MIPS64 (Intrinsics)
###############################################################################
elseif(CPU_TYPE STREQUAL "loongson" OR CPU_TYPE MATCHES "^mips64")
set(CMAKE_REQUIRED_FLAGS -Wa,-mloongson-mmi,-mloongson-ext)
check_c_source_compiles("
#if !(defined(__mips__) && __mips_isa_rev < 6)
#error \"Loongson MMI can't work with MIPS Release 6+\"
#endif
int main(void) {
int c = 0, a = 0, b = 0;
asm (
\"paddb %0, %1, %2\"
: \"=f\" (c)
: \"f\" (a), \"f\" (b)
);
return c;
}" HAVE_MMI)
unset(CMAKE_REQUIRED_FLAGS)
if(NOT HAVE_MMI)
simd_fail("SIMD extensions not available for this CPU")
return()
endif()
set(SIMD_SOURCES mips64/jccolor-mmi.c mips64/jcgray-mmi.c mips64/jcsample-mmi.c
mips64/jdcolor-mmi.c mips64/jdmerge-mmi.c mips64/jdsample-mmi.c
mips64/jfdctfst-mmi.c mips64/jfdctint-mmi.c mips64/jidctfst-mmi.c
mips64/jidctint-mmi.c mips64/jquanti-mmi.c)
if(CMAKE_COMPILER_IS_GNUCC)
foreach(file ${SIMD_SOURCES})
set_property(SOURCE ${file} APPEND_STRING PROPERTY COMPILE_FLAGS
" -fno-strict-aliasing")
endforeach()
endif()
foreach(file ${SIMD_SOURCES})
set_property(SOURCE ${file} APPEND_STRING PROPERTY COMPILE_FLAGS
" -Wa,-mloongson-mmi,-mloongson-ext")
endforeach()
add_library(simd OBJECT ${SIMD_SOURCES} mips64/jsimd.c)
if(CMAKE_POSITION_INDEPENDENT_CODE OR ENABLE_SHARED)
set_target_properties(simd PROPERTIES POSITION_INDEPENDENT_CODE 1)
endif()
###############################################################################
# PowerPC (Intrinsics)
###############################################################################
elseif(CPU_TYPE STREQUAL "powerpc")
set(CMAKE_REQUIRED_FLAGS -maltivec)
check_c_source_compiles("
#include <altivec.h>
int main(void) {
__vector int vi = { 0, 0, 0, 0 };
int i[4];
vec_st(vi, 0, i);
return i[0];
}" HAVE_ALTIVEC)
unset(CMAKE_REQUIRED_FLAGS)
if(NOT HAVE_ALTIVEC)
simd_fail("SIMD extensions not available for this CPU (PowerPC SPE)")
return()
endif()
set(SIMD_SOURCES powerpc/jccolor-altivec.c powerpc/jcgray-altivec.c
powerpc/jcsample-altivec.c powerpc/jdcolor-altivec.c
powerpc/jdmerge-altivec.c powerpc/jdsample-altivec.c
powerpc/jfdctfst-altivec.c powerpc/jfdctint-altivec.c
powerpc/jidctfst-altivec.c powerpc/jidctint-altivec.c
powerpc/jquanti-altivec.c)
set_source_files_properties(${SIMD_SOURCES} PROPERTIES
COMPILE_FLAGS -maltivec)
if(UNIX)
include(CheckSymbolExists)
check_symbol_exists(getauxval sys/auxv.h HAVE_GETAUXVAL)
if(HAVE_GETAUXVAL)
set_source_files_properties(powerpc/jsimd.c PROPERTIES
COMPILE_DEFINITIONS HAVE_GETAUXVAL)
endif()
check_symbol_exists(elf_aux_info sys/auxv.h HAVE_ELF_AUX_INFO)
if(HAVE_ELF_AUX_INFO)
set_source_files_properties(powerpc/jsimd.c PROPERTIES
COMPILE_DEFINITIONS HAVE_ELF_AUX_INFO)
endif()
endif()
add_library(simd OBJECT ${SIMD_SOURCES} powerpc/jsimd.c)
if(CMAKE_POSITION_INDEPENDENT_CODE OR ENABLE_SHARED)
set_target_properties(simd PROPERTIES POSITION_INDEPENDENT_CODE 1)
endif()
###############################################################################
# None
###############################################################################
else()
simd_fail("SIMD extensions not available for this CPU (${CMAKE_SYSTEM_PROCESSOR})")
endif() # CPU_TYPE
if(WITH_SIMD AND ENABLE_STATIC)
add_executable(simdcoverage simdcoverage.c)
target_link_libraries(simdcoverage jpeg-static)
endif()
lib/libjpeg-turbo/simd/jsimd.h
Executable
+1256
View File
File diff suppressed because it is too large Load Diff
lib/libjpeg-turbo/simd/simdcoverage.c
Executable
+121
View File
@@ -0,0 +1,121 @@
/*
* Copyright (C) 2025, D. R. Commander. All Rights Reserved.
*
* This software is provided 'as-is', without any express or implied
* warranty. In no event will the authors be held liable for any damages
* arising from the use of this software.
*
* Permission is granted to anyone to use this software for any purpose,
* including commercial applications, and to alter it and redistribute it
* freely, subject to the following restrictions:
*
* 1. The origin of this software must not be misrepresented; you must not
* claim that you wrote the original software. If you use this software
* in a product, an acknowledgment in the product documentation would be
* appreciated but is not required.
* 2. Altered source versions must be plainly marked as such, and must not be
* misrepresented as being the original software.
* 3. This notice may not be removed or altered from any source distribution.
*/
#include <stdio.h>
#define JPEG_INTERNALS
#include "../src/jinclude.h"
#include "../src/jpeglib.h"
#include "../src/jdct.h"
#include "../src/jsimddct.h"
#include "../src/jsimd.h"
#include "../src/jpegapicomp.h"
#define C_COVERAGE_TEST(f) \
printf(#f " -- %s\n", f() ? "YES" : "no")
#define D_COVERAGE_TEST(f) \
printf(#f " -- %s\n", f() ? "YES" : "no")
int main(void)
{
struct jpeg_compress_struct cinfo;
struct jpeg_decompress_struct dinfo;
struct jpeg_error_mgr jerr;
cinfo.err = jpeg_std_error(&jerr);
jpeg_create_compress(&cinfo);
cinfo.input_components = 3;
cinfo.in_color_space = JCS_RGB;
jpeg_set_defaults(&cinfo);
jinit_color_converter(&cinfo);
jinit_downsampler(&cinfo);
jinit_huff_encoder(&cinfo);
dinfo.err = jpeg_std_error(&jerr);
jpeg_create_decompress(&dinfo);
dinfo.num_components = 3;
dinfo.jpeg_color_space = JCS_YCbCr;
dinfo.out_color_space = JCS_RGB;
jinit_color_deconverter(&dinfo);
dinfo.comp_info = (jpeg_component_info *)(*dinfo.mem->alloc_small)
((j_common_ptr)&dinfo, JPOOL_IMAGE,
dinfo.num_components * sizeof(jpeg_component_info));
dinfo.comp_info[0].component_id = 0;
dinfo.comp_info[0].h_samp_factor = dinfo.comp_info[0].v_samp_factor = 1;
dinfo.comp_info[1].component_id = 1;
dinfo.comp_info[1].h_samp_factor = dinfo.comp_info[1].v_samp_factor = 1;
dinfo.comp_info[2].component_id = 2;
dinfo.comp_info[2].h_samp_factor = dinfo.comp_info[2].v_samp_factor = 1;
dinfo._min_DCT_scaled_size = DCTSIZE;
jinit_upsampler(&dinfo);
jinit_inverse_dct(&dinfo);
C_COVERAGE_TEST(jsimd_can_rgb_ycc);
C_COVERAGE_TEST(jsimd_can_rgb_gray);
D_COVERAGE_TEST(jsimd_can_ycc_rgb);
D_COVERAGE_TEST(jsimd_can_ycc_rgb565);
C_COVERAGE_TEST(jsimd_can_h2v1_downsample);
C_COVERAGE_TEST(jsimd_can_h2v2_downsample);
#ifdef __mips__
C_COVERAGE_TEST(jsimd_can_h2v2_smooth_downsample);
#endif
D_COVERAGE_TEST(jsimd_can_h2v1_upsample);
D_COVERAGE_TEST(jsimd_can_h2v2_upsample);
#ifdef __mips__
D_COVERAGE_TEST(jsimd_can_int_upsample);
#endif
D_COVERAGE_TEST(jsimd_can_h2v1_fancy_upsample);
D_COVERAGE_TEST(jsimd_can_h2v2_fancy_upsample);
#if defined(__arm__) || defined(__aarch64__) || defined(_M_ARM) || \
defined(_M_ARM64) || defined(_M_ARM64EC)
D_COVERAGE_TEST(jsimd_can_h1v2_fancy_upsample);
#endif
D_COVERAGE_TEST(jsimd_can_h2v1_merged_upsample);
D_COVERAGE_TEST(jsimd_can_h2v2_merged_upsample);
C_COVERAGE_TEST(jsimd_can_convsamp);
C_COVERAGE_TEST(jsimd_can_convsamp_float);
C_COVERAGE_TEST(jsimd_can_fdct_islow);
C_COVERAGE_TEST(jsimd_can_fdct_ifast);
C_COVERAGE_TEST(jsimd_can_fdct_float);
C_COVERAGE_TEST(jsimd_can_quantize);
C_COVERAGE_TEST(jsimd_can_quantize_float);
D_COVERAGE_TEST(jsimd_can_idct_islow);
D_COVERAGE_TEST(jsimd_can_idct_ifast);
D_COVERAGE_TEST(jsimd_can_idct_float);
D_COVERAGE_TEST(jsimd_can_idct_2x2);
D_COVERAGE_TEST(jsimd_can_idct_4x4);
#ifdef __mips__
D_COVERAGE_TEST(jsimd_can_idct_6x6);
D_COVERAGE_TEST(jsimd_can_idct_12x12);
#endif
C_COVERAGE_TEST(jsimd_can_huff_encode_one_block);
C_COVERAGE_TEST(jsimd_can_encode_mcu_AC_first_prepare);
C_COVERAGE_TEST(jsimd_can_encode_mcu_AC_refine_prepare);
jpeg_abort_compress(&cinfo);
jpeg_destroy_compress(&cinfo);
jpeg_abort_decompress(&dinfo);
jpeg_destroy_decompress(&dinfo);
return 0;
}
lib/libjpeg-turbo/simd/x86_64/jccolext-sse2.asm
Executable
+541
View File
@@ -0,0 +1,541 @@
;
; Colorspace conversion (64-bit SSE2)
;
; Copyright (C) 2009, 2016, 2024-2025, D. R. Commander.
; Copyright (C) 2018, Matthias Räncker.
; Copyright (C) 2023, Aliaksiej Kandracienka.
;
; Based on the x86 SIMD extension for IJG JPEG library
; Copyright (C) 1999-2006, MIYASAKA Masaru.
; For conditions of distribution and use, see copyright notice in jsimdext.inc
;
; This file should be assembled with NASM (Netwide Assembler) or Yasm.
%include "jcolsamp.inc"
; --------------------------------------------------------------------------
;
; Convert some rows of samples to the JPEG colorspace.
;
; GLOBAL(void)
; jsimd_rgb_ycc_convert_sse2(JDIMENSION img_width, JSAMPARRAY input_buf,
; JSAMPIMAGE output_buf, JDIMENSION output_row,
; int num_rows)
;
; r10d = JDIMENSION img_width
; r11 = JSAMPARRAY input_buf
; r12 = JSAMPIMAGE output_buf
; r13d = JDIMENSION output_row
; r14d = int num_rows
%define wk(i) r15 - (WK_NUM - (i)) * SIZEOF_XMMWORD ; xmmword wk[WK_NUM]
%define WK_NUM 8
align 32
GLOBAL_FUNCTION(jsimd_rgb_ycc_convert_sse2)
EXTN(jsimd_rgb_ycc_convert_sse2):
ENDBR64
push rbp
mov rbp, rsp
push r15
and rsp, byte (-SIZEOF_XMMWORD) ; align to 128 bits
; Allocate stack space for wk array. r15 is used to access it.
mov r15, rsp
sub rsp, (SIZEOF_XMMWORD * WK_NUM)
COLLECT_ARGS 5
push rbx
mov ecx, r10d
test rcx, rcx
jz near .return
push rcx
mov rsi, r12
mov ecx, r13d
mov rdip, JSAMPARRAY [rsi + 0 * SIZEOF_JSAMPARRAY]
mov rbxp, JSAMPARRAY [rsi + 1 * SIZEOF_JSAMPARRAY]
mov rdxp, JSAMPARRAY [rsi + 2 * SIZEOF_JSAMPARRAY]
lea rdi, [rdi + rcx * SIZEOF_JSAMPROW]
lea rbx, [rbx + rcx * SIZEOF_JSAMPROW]
lea rdx, [rdx + rcx * SIZEOF_JSAMPROW]
pop rcx
mov rsi, r11
mov eax, r14d
test rax, rax
jle near .return
.rowloop:
push rdx
push rbx
push rdi
push rsi
push rcx ; col
mov rsip, JSAMPROW [rsi] ; inptr
mov rdip, JSAMPROW [rdi] ; outptr0
mov rbxp, JSAMPROW [rbx] ; outptr1
mov rdxp, JSAMPROW [rdx] ; outptr2
cmp rcx, byte SIZEOF_XMMWORD
jae near .columnloop
%if RGB_PIXELSIZE == 3 ; ---------------
.column_ld1:
push rax
push rdx
lea rcx, [rcx + rcx * 2] ; imul ecx, RGB_PIXELSIZE
test cl, SIZEOF_BYTE
jz short .column_ld2
sub rcx, byte SIZEOF_BYTE
movzx rax, byte [rsi + rcx]
.column_ld2:
test cl, SIZEOF_WORD
jz short .column_ld4
sub rcx, byte SIZEOF_WORD
movzx rdx, word [rsi + rcx]
shl rax, WORD_BIT
or rax, rdx
.column_ld4:
movd xmmA, eax
pop rdx
pop rax
test cl, SIZEOF_DWORD
jz short .column_ld8
sub rcx, byte SIZEOF_DWORD
movd xmmF, XMM_DWORD [rsi + rcx]
pslldq xmmA, SIZEOF_DWORD
por xmmA, xmmF
.column_ld8:
test cl, SIZEOF_MMWORD
jz short .column_ld16
sub rcx, byte SIZEOF_MMWORD
movq xmmB, XMM_MMWORD [rsi + rcx]
pslldq xmmA, SIZEOF_MMWORD
por xmmA, xmmB
.column_ld16:
test cl, SIZEOF_XMMWORD
jz short .column_ld32
movdqa xmmF, xmmA
movdqu xmmA, XMMWORD [rsi + 0 * SIZEOF_XMMWORD]
mov rcx, SIZEOF_XMMWORD
jmp short .rgb_ycc_cnv
.column_ld32:
test cl, 2 * SIZEOF_XMMWORD
mov rcx, SIZEOF_XMMWORD
jz short .rgb_ycc_cnv
movdqa xmmB, xmmA
movdqu xmmA, XMMWORD [rsi + 0 * SIZEOF_XMMWORD]
movdqu xmmF, XMMWORD [rsi + 1 * SIZEOF_XMMWORD]
jmp short .rgb_ycc_cnv
.columnloop:
movdqu xmmA, XMMWORD [rsi + 0 * SIZEOF_XMMWORD]
movdqu xmmF, XMMWORD [rsi + 1 * SIZEOF_XMMWORD]
movdqu xmmB, XMMWORD [rsi + 2 * SIZEOF_XMMWORD]
.rgb_ycc_cnv:
; NOTE: The values of RGB_RED, RGB_GREEN, and RGB_BLUE determine the
; mapping of components A, B, and C to red, green, and blue.
;
; xmmA = (A0 B0 C0 A1 B1 C1 A2 B2 C2 A3 B3 C3 A4 B4 C4 A5)
; xmmF = (B5 C5 A6 B6 C6 A7 B7 C7 A8 B8 C8 A9 B9 C9 Aa Ba)
; xmmB = (Ca Ab Bb Cb Ac Bc Cc Ad Bd Cd Ae Be Ce Af Bf Cf)
movdqa xmmG, xmmA
pslldq xmmA, 8
; xmmA = (-- -- -- -- -- -- -- -- A0 B0 C0 A1 B1 C1 A2 B2)
psrldq xmmG, 8
; xmmG = (C2 A3 B3 C3 A4 B4 C4 A5 -- -- -- -- -- -- -- --)
punpckhbw xmmA, xmmF
; xmmA = (A0 A8 B0 B8 C0 C8 A1 A9 B1 B9 C1 C9 A2 Aa B2 Ba)
pslldq xmmF, 8
; xmmF = (-- -- -- -- -- -- -- -- B5 C5 A6 B6 C6 A7 B7 C7)
punpcklbw xmmG, xmmB
; xmmG = (C2 Ca A3 Ab B3 Bb C3 Cb A4 Ac B4 Bc C4 Cc A5 Ad)
punpckhbw xmmF, xmmB
; xmmF = (B5 Bd C5 Cd A6 Ae B6 Be C6 Ce A7 Af B7 Bf C7 Cf)
movdqa xmmD, xmmA
pslldq xmmA, 8
; xmmA = (-- -- -- -- -- -- -- -- A0 A8 B0 B8 C0 C8 A1 A9)
psrldq xmmD, 8
; xmmD = (B1 B9 C1 C9 A2 Aa B2 Ba -- -- -- -- -- -- -- --)
punpckhbw xmmA, xmmG
; xmmA = (A0 A4 A8 Ac B0 B4 B8 Bc C0 C4 C8 Cc A1 A5 A9 Ad)
pslldq xmmG, 8
; xmmG = (-- -- -- -- -- -- -- -- C2 Ca A3 Ab B3 Bb C3 Cb)
punpcklbw xmmD, xmmF
; xmmD = (B1 B5 B9 Bd C1 C5 C9 Cd A2 A6 Aa Ae B2 B6 Ba Be)
punpckhbw xmmG, xmmF
; xmmG = (C2 C6 Ca Ce A3 A7 Ab Af B3 B7 Bb Bf C3 C7 Cb Cf)
movdqa xmmE, xmmA
pslldq xmmA, 8
; xmmA = (-- -- -- -- -- -- -- -- A0 A4 A8 Ac B0 B4 B8 Bc)
psrldq xmmE, 8
; xmmE = (C0 C4 C8 Cc A1 A5 A9 Ad -- -- -- -- -- -- -- --)
punpckhbw xmmA, xmmD
; xmmA = (A0 A2 A4 A6 A8 Aa Ac Ae B0 B2 B4 B6 B8 Ba Bc Be)
pslldq xmmD, 8
; xmmD = (-- -- -- -- -- -- -- -- B1 B5 B9 Bd C1 C5 C9 Cd)
punpcklbw xmmE, xmmG
; xmmE = (C0 C2 C4 C6 C8 Ca Cc Ce A1 A3 A5 A7 A9 Ab Ad Af)
punpckhbw xmmD, xmmG
; xmmD = (B1 B3 B5 B7 B9 Bb Bd Bf C1 C3 C5 C7 C9 Cb Cd Cf)
pxor xmmH, xmmH
movdqa xmmC, xmmA
punpcklbw xmmA, xmmH ; xmmA = (A0 A2 A4 A6 A8 Aa Ac Ae) = AE
punpckhbw xmmC, xmmH ; xmmC = (B0 B2 B4 B6 B8 Ba Bc Be) = BE
movdqa xmmB, xmmE
punpcklbw xmmE, xmmH ; xmmE = (C0 C2 C4 C6 C8 Ca Cc Ce) = CE
punpckhbw xmmB, xmmH ; xmmB = (A1 A3 A5 A7 A9 Ab Ad Af) = AO
movdqa xmmF, xmmD
punpcklbw xmmD, xmmH ; xmmD = (B1 B3 B5 B7 B9 Bb Bd Bf) = BO
punpckhbw xmmF, xmmH ; xmmF = (C1 C3 C5 C7 C9 Cb Cd Cf) = CO
%else ; RGB_PIXELSIZE == 4 ; -----------
.column_ld1:
test cl, SIZEOF_XMMWORD / 16
jz short .column_ld2
sub rcx, byte SIZEOF_XMMWORD / 16
movd xmmA, XMM_DWORD [rsi + rcx * RGB_PIXELSIZE]
.column_ld2:
test cl, SIZEOF_XMMWORD / 8
jz short .column_ld4
sub rcx, byte SIZEOF_XMMWORD / 8
movq xmmE, XMM_MMWORD [rsi + rcx * RGB_PIXELSIZE]
pslldq xmmA, SIZEOF_MMWORD
por xmmA, xmmE
.column_ld4:
test cl, SIZEOF_XMMWORD / 4
jz short .column_ld8
sub rcx, byte SIZEOF_XMMWORD / 4
movdqa xmmE, xmmA
movdqu xmmA, XMMWORD [rsi + rcx * RGB_PIXELSIZE]
.column_ld8:
test cl, SIZEOF_XMMWORD / 2
mov rcx, SIZEOF_XMMWORD
jz short .rgb_ycc_cnv
movdqa xmmF, xmmA
movdqa xmmH, xmmE
movdqu xmmA, XMMWORD [rsi + 0 * SIZEOF_XMMWORD]
movdqu xmmE, XMMWORD [rsi + 1 * SIZEOF_XMMWORD]
jmp short .rgb_ycc_cnv
.columnloop:
movdqu xmmA, XMMWORD [rsi + 0 * SIZEOF_XMMWORD]
movdqu xmmE, XMMWORD [rsi + 1 * SIZEOF_XMMWORD]
movdqu xmmF, XMMWORD [rsi + 2 * SIZEOF_XMMWORD]
movdqu xmmH, XMMWORD [rsi + 3 * SIZEOF_XMMWORD]
.rgb_ycc_cnv:
; NOTE: The values of RGB_RED, RGB_GREEN, and RGB_BLUE determine the
; mapping of components A, B, C, and D to red, green, and blue.
;
; xmmA = (A0 B0 C0 D0 A1 B1 C1 D1 A2 B2 C2 D2 A3 B3 C3 D3)
; xmmE = (A4 B4 C4 D4 A5 B5 C5 D5 A6 B6 C6 D6 A7 B7 C7 D7)
; xmmF = (A8 B8 C8 D8 A9 B9 C9 D9 Aa Ba Ca Da Ab Bb Cb Db)
; xmmH = (Ac Bc Cc Dc Ad Bd Cd Dd Ae Be Ce De Af Bf Cf Df)
movdqa xmmD, xmmA
punpcklbw xmmA, xmmE
; xmmA = (A0 A4 B0 B4 C0 C4 D0 D4 A1 A5 B1 B5 C1 C5 D1 D5)
punpckhbw xmmD, xmmE
; xmmD = (A2 A6 B2 B6 C2 C6 D2 D6 A3 A7 B3 B7 C3 C7 D3 D7)
movdqa xmmC, xmmF
punpcklbw xmmF, xmmH
; xmmF = (A8 Ac B8 Bc C8 Cc D8 Dc A9 Ad B9 Bd C9 Cd D9 Dd)
punpckhbw xmmC, xmmH
; xmmC = (Aa Ae Ba Be Ca Ce Da De Ab Af Bb Bf Cb Cf Db Df)
movdqa xmmB, xmmA
punpcklwd xmmA, xmmF
; xmmA = (A0 A4 A8 Ac B0 B4 B8 Bc C0 C4 C8 Cc D0 D4 D8 Dc)
punpckhwd xmmB, xmmF
; xmmB = (A1 A5 A9 Ad B1 B5 B9 Bd C1 C5 C9 Cd D1 D5 D9 Dd)
movdqa xmmG, xmmD
punpcklwd xmmD, xmmC
; xmmD = (A2 A6 Aa Ae B2 B6 Ba Be C2 C6 Ca Ce D2 D6 Da De)
punpckhwd xmmG, xmmC
; xmmG = (A3 A7 Ab Af B3 B7 Bb Bf C3 C7 Cb Cf D3 D7 Db Df)
movdqa xmmE, xmmA
punpcklbw xmmA, xmmD
; xmmA = (A0 A2 A4 A6 A8 Aa Ac Ae B0 B2 B4 B6 B8 Ba Bc Be)
punpckhbw xmmE, xmmD
; xmmE = (C0 C2 C4 C6 C8 Ca Cc Ce D0 D2 D4 D6 D8 Da Dc De)
movdqa xmmH, xmmB
punpcklbw xmmB, xmmG
; xmmB = (A1 A3 A5 A7 A9 Ab Ad Af B1 B3 B5 B7 B9 Bb Bd Bf)
punpckhbw xmmH, xmmG
; xmmH = (C1 C3 C5 C7 C9 Cb Cd Cf D1 D3 D5 D7 D9 Db Dd Df)
pxor xmmF, xmmF
movdqa xmmC, xmmA
punpcklbw xmmA, xmmF ; xmmA = (A0 A2 A4 A6 A8 Aa Ac Ae) = AE
punpckhbw xmmC, xmmF ; xmmC = (B0 B2 B4 B6 B8 Ba Bc Be) = BE
movdqa xmmD, xmmB
punpcklbw xmmB, xmmF ; xmmB = (A1 A3 A5 A7 A9 Ab Ad Af) = AO
punpckhbw xmmD, xmmF ; xmmD = (B1 B3 B5 B7 B9 Bb Bd Bf) = BO
movdqa xmmG, xmmE
punpcklbw xmmE, xmmF ; xmmE = (C0 C2 C4 C6 C8 Ca Cc Ce) = CE
punpckhbw xmmG, xmmF ; xmmG = (D0 D2 D4 D6 D8 Da Dc De) = DE
punpcklbw xmmF, xmmH
punpckhbw xmmH, xmmH
psrlw xmmF, BYTE_BIT ; xmmF = (C1 C3 C5 C7 C9 Cb Cd Cf) = CO
psrlw xmmH, BYTE_BIT ; xmmH = (D1 D3 D5 D7 D9 Db Dd Df) = DO
%endif ; RGB_PIXELSIZE ; ---------------
; xmm0 = (R0 R2 R4 R6 R8 Ra Rc Re) = RE
; xmm2 = (G0 G2 G4 G6 G8 Ga Gc Ge) = GE
; xmm4 = (B0 B2 B4 B6 B8 Ba Bc Be) = BE
; xmm1 = (R1 R3 R5 R7 R9 Rb Rd Rf) = RO
; xmm3 = (G1 G3 G5 G7 G9 Gb Gd Gf) = GO
; xmm5 = (B1 B3 B5 B7 B9 Bb Bd Bf) = BO
;
; (Original)
; Y = 0.29900 * R + 0.58700 * G + 0.11400 * B
; Cb = -0.16874 * R - 0.33126 * G + 0.50000 * B + CENTERJSAMPLE
; Cr = 0.50000 * R - 0.41869 * G - 0.08131 * B + CENTERJSAMPLE
;
; (This implementation)
; Y = 0.29900 * R + 0.33700 * G + 0.11400 * B + 0.25000 * G
; Cb = -0.16874 * R - 0.33126 * G + 0.50000 * B + CENTERJSAMPLE
; Cr = 0.50000 * R - 0.41869 * G - 0.08131 * B + CENTERJSAMPLE
movdqa XMMWORD [wk(0)], xmm0 ; wk(0) = RE
movdqa XMMWORD [wk(1)], xmm1 ; wk(1) = RO
movdqa XMMWORD [wk(2)], xmm4 ; wk(2) = BE
movdqa XMMWORD [wk(3)], xmm5 ; wk(3) = BO
movdqa xmm6, xmm1
punpcklwd xmm1, xmm3
punpckhwd xmm6, xmm3
movdqa xmm7, xmm1
movdqa xmm4, xmm6
pmaddwd xmm1, [rel PW_F0299_F0337]
; xmm1 = ROL * FIX(0.299) + GOL * FIX(0.337)
pmaddwd xmm6, [rel PW_F0299_F0337]
; xmm6 = ROH * FIX(0.299) + GOH * FIX(0.337)
pmaddwd xmm7, [rel PW_MF016_MF033]
; xmm7 = ROL * -FIX(0.168) + GOL * -FIX(0.331)
pmaddwd xmm4, [rel PW_MF016_MF033]
; xmm4 = ROH * -FIX(0.168) + GOH * -FIX(0.331)
movdqa XMMWORD [wk(4)], xmm1
; wk(4) = ROL * FIX(0.299) + GOL * FIX(0.337)
movdqa XMMWORD [wk(5)], xmm6
; wk(5) = ROH * FIX(0.299) + GOH * FIX(0.337)
pxor xmm1, xmm1
pxor xmm6, xmm6
punpcklwd xmm1, xmm5 ; xmm1 = BOL
punpckhwd xmm6, xmm5 ; xmm6 = BOH
psrld xmm1, 1 ; xmm1 = BOL * FIX(0.500)
psrld xmm6, 1 ; xmm6 = BOH * FIX(0.500)
movdqa xmm5, [rel PD_ONEHALFM1_CJ] ; xmm5 = [PD_ONEHALFM1_CJ]
paddd xmm7, xmm1
paddd xmm4, xmm6
paddd xmm7, xmm5
paddd xmm4, xmm5
psrld xmm7, SCALEBITS ; xmm7 = CbOL
psrld xmm4, SCALEBITS ; xmm4 = CbOH
packssdw xmm7, xmm4 ; xmm7 = CbO
movdqa xmm1, XMMWORD [wk(2)] ; xmm1 = BE
movdqa xmm6, xmm0
punpcklwd xmm0, xmm2
punpckhwd xmm6, xmm2
movdqa xmm5, xmm0
movdqa xmm4, xmm6
pmaddwd xmm0, [rel PW_F0299_F0337]
; xmm0 = REL * FIX(0.299) + GEL * FIX(0.337)
pmaddwd xmm6, [rel PW_F0299_F0337]
; xmm6 = REH * FIX(0.299) + GEH * FIX(0.337)
pmaddwd xmm5, [rel PW_MF016_MF033]
; xmm5 = REL * -FIX(0.168) + GEL * -FIX(0.331)
pmaddwd xmm4, [rel PW_MF016_MF033]
; xmm4 = REH * -FIX(0.168) + GEH * -FIX(0.331)
movdqa XMMWORD [wk(6)], xmm0
; wk(6) = REL * FIX(0.299) + GEL * FIX(0.337)
movdqa XMMWORD [wk(7)], xmm6
; wk(7) = REH * FIX(0.299) + GEH * FIX(0.337)
pxor xmm0, xmm0
pxor xmm6, xmm6
punpcklwd xmm0, xmm1 ; xmm0 = BEL
punpckhwd xmm6, xmm1 ; xmm6 = BEH
psrld xmm0, 1 ; xmm0 = BEL * FIX(0.500)
psrld xmm6, 1 ; xmm6 = BEH * FIX(0.500)
movdqa xmm1, [rel PD_ONEHALFM1_CJ] ; xmm1 = [PD_ONEHALFM1_CJ]
paddd xmm5, xmm0
paddd xmm4, xmm6
paddd xmm5, xmm1
paddd xmm4, xmm1
psrld xmm5, SCALEBITS ; xmm5 = CbEL
psrld xmm4, SCALEBITS ; xmm4 = CbEH
packssdw xmm5, xmm4 ; xmm5 = CbE
psllw xmm7, BYTE_BIT
por xmm5, xmm7 ; xmm5 = Cb
movdqa XMMWORD [rbx], xmm5 ; Save Cb
movdqa xmm0, XMMWORD [wk(3)] ; xmm0 = BO
movdqa xmm6, XMMWORD [wk(2)] ; xmm6 = BE
movdqa xmm1, XMMWORD [wk(1)] ; xmm1 = RO
movdqa xmm4, xmm0
punpcklwd xmm0, xmm3
punpckhwd xmm4, xmm3
movdqa xmm7, xmm0
movdqa xmm5, xmm4
pmaddwd xmm0, [rel PW_F0114_F0250]
; xmm0 = BOL * FIX(0.114) + GOL * FIX(0.250)
pmaddwd xmm4, [rel PW_F0114_F0250]
; xmm4 = BOH * FIX(0.114) + GOH * FIX(0.250)
pmaddwd xmm7, [rel PW_MF008_MF041]
; xmm7 = BOL * -FIX(0.081) + GOL * -FIX(0.418)
pmaddwd xmm5, [rel PW_MF008_MF041]
; xmm5 = BOH * -FIX(0.081) + GOH * -FIX(0.418)
movdqa xmm3, [rel PD_ONEHALF] ; xmm3 = [PD_ONEHALF]
paddd xmm0, XMMWORD [wk(4)]
paddd xmm4, XMMWORD [wk(5)]
paddd xmm0, xmm3
paddd xmm4, xmm3
psrld xmm0, SCALEBITS ; xmm0 = YOL
psrld xmm4, SCALEBITS ; xmm4 = YOH
packssdw xmm0, xmm4 ; xmm0 = YO
pxor xmm3, xmm3
pxor xmm4, xmm4
punpcklwd xmm3, xmm1 ; xmm3 = ROL
punpckhwd xmm4, xmm1 ; xmm4 = ROH
psrld xmm3, 1 ; xmm3 = ROL * FIX(0.500)
psrld xmm4, 1 ; xmm4 = ROH * FIX(0.500)
movdqa xmm1, [rel PD_ONEHALFM1_CJ] ; xmm1 = [PD_ONEHALFM1_CJ]
paddd xmm7, xmm3
paddd xmm5, xmm4
paddd xmm7, xmm1
paddd xmm5, xmm1
psrld xmm7, SCALEBITS ; xmm7 = CrOL
psrld xmm5, SCALEBITS ; xmm5 = CrOH
packssdw xmm7, xmm5 ; xmm7 = CrO
movdqa xmm3, XMMWORD [wk(0)] ; xmm3 = RE
movdqa xmm4, xmm6
punpcklwd xmm6, xmm2
punpckhwd xmm4, xmm2
movdqa xmm1, xmm6
movdqa xmm5, xmm4
pmaddwd xmm6, [rel PW_F0114_F0250]
; xmm6 = BEL * FIX(0.114) + GEL * FIX(0.250)
pmaddwd xmm4, [rel PW_F0114_F0250]
; xmm4 = BEH * FIX(0.114) + GEH * FIX(0.250)
pmaddwd xmm1, [rel PW_MF008_MF041]
; xmm1 = BEL * -FIX(0.081) + GEL * -FIX(0.418)
pmaddwd xmm5, [rel PW_MF008_MF041]
; xmm5 = BEH * -FIX(0.081) + GEH * -FIX(0.418)
movdqa xmm2, [rel PD_ONEHALF] ; xmm2 = [PD_ONEHALF]
paddd xmm6, XMMWORD [wk(6)]
paddd xmm4, XMMWORD [wk(7)]
paddd xmm6, xmm2
paddd xmm4, xmm2
psrld xmm6, SCALEBITS ; xmm6 = YEL
psrld xmm4, SCALEBITS ; xmm4 = YEH
packssdw xmm6, xmm4 ; xmm6 = YE
psllw xmm0, BYTE_BIT
por xmm6, xmm0 ; xmm6 = Y
movdqa XMMWORD [rdi], xmm6 ; Save Y
pxor xmm2, xmm2
pxor xmm4, xmm4
punpcklwd xmm2, xmm3 ; xmm2 = REL
punpckhwd xmm4, xmm3 ; xmm4 = REH
psrld xmm2, 1 ; xmm2 = REL * FIX(0.500)
psrld xmm4, 1 ; xmm4 = REH * FIX(0.500)
movdqa xmm0, [rel PD_ONEHALFM1_CJ] ; xmm0 = [PD_ONEHALFM1_CJ]
paddd xmm1, xmm2
paddd xmm5, xmm4
paddd xmm1, xmm0
paddd xmm5, xmm0
psrld xmm1, SCALEBITS ; xmm1 = CrEL
psrld xmm5, SCALEBITS ; xmm5 = CrEH
packssdw xmm1, xmm5 ; xmm1 = CrE
psllw xmm7, BYTE_BIT
por xmm1, xmm7 ; xmm1 = Cr
movdqa XMMWORD [rdx], xmm1 ; Save Cr
sub rcx, byte SIZEOF_XMMWORD
add rsi, byte RGB_PIXELSIZE * SIZEOF_XMMWORD ; inptr
add rdi, byte SIZEOF_XMMWORD ; outptr0
add rbx, byte SIZEOF_XMMWORD ; outptr1
add rdx, byte SIZEOF_XMMWORD ; outptr2
cmp rcx, byte SIZEOF_XMMWORD
jae near .columnloop
test rcx, rcx
jnz near .column_ld1
pop rcx ; col
pop rsi
pop rdi
pop rbx
pop rdx
add rsi, byte SIZEOF_JSAMPROW ; input_buf
add rdi, byte SIZEOF_JSAMPROW
add rbx, byte SIZEOF_JSAMPROW
add rdx, byte SIZEOF_JSAMPROW
dec rax ; num_rows
jg near .rowloop
.return:
pop rbx
UNCOLLECT_ARGS 5
lea rsp, [rbp - 8]
pop r15
pop rbp
ret
; For some reason, the OS X linker does not honor the request to align the
; segment unless we do this.
align 32
lib/libjpeg-turbo/simd/x86_64/jccolor-avx2.asm
Executable
+117
View File
@@ -0,0 +1,117 @@
;
; Colorspace conversion (64-bit AVX2)
;
; Copyright (C) 2009, 2016, 2024, D. R. Commander.
; Copyright (C) 2015, Intel Corporation.
;
; Based on the x86 SIMD extension for IJG JPEG library
; Copyright (C) 1999-2006, MIYASAKA Masaru.
; For conditions of distribution and use, see copyright notice in jsimdext.inc
;
; This file should be assembled with NASM (Netwide Assembler) or Yasm.
%include "jsimdext.inc"
; --------------------------------------------------------------------------
%define SCALEBITS 16
F_0_081 equ 5329 ; FIX(0.08131)
F_0_114 equ 7471 ; FIX(0.11400)
F_0_168 equ 11059 ; FIX(0.16874)
F_0_250 equ 16384 ; FIX(0.25000)
F_0_299 equ 19595 ; FIX(0.29900)
F_0_331 equ 21709 ; FIX(0.33126)
F_0_418 equ 27439 ; FIX(0.41869)
F_0_587 equ 38470 ; FIX(0.58700)
F_0_337 equ (F_0_587 - F_0_250) ; FIX(0.58700) - FIX(0.25000)
; --------------------------------------------------------------------------
SECTION SEG_CONST
ALIGNZ 32
GLOBAL_DATA(jconst_rgb_ycc_convert_avx2)
EXTN(jconst_rgb_ycc_convert_avx2):
PW_F0299_F0337 times 8 dw F_0_299, F_0_337
PW_F0114_F0250 times 8 dw F_0_114, F_0_250
PW_MF016_MF033 times 8 dw -F_0_168, -F_0_331
PW_MF008_MF041 times 8 dw -F_0_081, -F_0_418
PD_ONEHALFM1_CJ times 8 dd (1 << (SCALEBITS - 1)) - 1 + \
(CENTERJSAMPLE << SCALEBITS)
PD_ONEHALF times 8 dd (1 << (SCALEBITS - 1))
ALIGNZ 32
; --------------------------------------------------------------------------
SECTION SEG_TEXT
BITS 64
%include "jccolext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_RGB_RED
%define RGB_GREEN EXT_RGB_GREEN
%define RGB_BLUE EXT_RGB_BLUE
%define RGB_PIXELSIZE EXT_RGB_PIXELSIZE
%define jsimd_rgb_ycc_convert_avx2 jsimd_extrgb_ycc_convert_avx2
%include "jccolext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_RGBX_RED
%define RGB_GREEN EXT_RGBX_GREEN
%define RGB_BLUE EXT_RGBX_BLUE
%define RGB_PIXELSIZE EXT_RGBX_PIXELSIZE
%define jsimd_rgb_ycc_convert_avx2 jsimd_extrgbx_ycc_convert_avx2
%include "jccolext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_BGR_RED
%define RGB_GREEN EXT_BGR_GREEN
%define RGB_BLUE EXT_BGR_BLUE
%define RGB_PIXELSIZE EXT_BGR_PIXELSIZE
%define jsimd_rgb_ycc_convert_avx2 jsimd_extbgr_ycc_convert_avx2
%include "jccolext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_BGRX_RED
%define RGB_GREEN EXT_BGRX_GREEN
%define RGB_BLUE EXT_BGRX_BLUE
%define RGB_PIXELSIZE EXT_BGRX_PIXELSIZE
%define jsimd_rgb_ycc_convert_avx2 jsimd_extbgrx_ycc_convert_avx2
%include "jccolext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_XBGR_RED
%define RGB_GREEN EXT_XBGR_GREEN
%define RGB_BLUE EXT_XBGR_BLUE
%define RGB_PIXELSIZE EXT_XBGR_PIXELSIZE
%define jsimd_rgb_ycc_convert_avx2 jsimd_extxbgr_ycc_convert_avx2
%include "jccolext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_XRGB_RED
%define RGB_GREEN EXT_XRGB_GREEN
%define RGB_BLUE EXT_XRGB_BLUE
%define RGB_PIXELSIZE EXT_XRGB_PIXELSIZE
%define jsimd_rgb_ycc_convert_avx2 jsimd_extxrgb_ycc_convert_avx2
%include "jccolext-avx2.asm"
lib/libjpeg-turbo/simd/x86_64/jccolor-sse2.asm
Executable
+116
View File
@@ -0,0 +1,116 @@
;
; Colorspace conversion (64-bit SSE2)
;
; Copyright (C) 2009, 2016, 2024, D. R. Commander.
;
; Based on the x86 SIMD extension for IJG JPEG library
; Copyright (C) 1999-2006, MIYASAKA Masaru.
; For conditions of distribution and use, see copyright notice in jsimdext.inc
;
; This file should be assembled with NASM (Netwide Assembler) or Yasm.
%include "jsimdext.inc"
; --------------------------------------------------------------------------
%define SCALEBITS 16
F_0_081 equ 5329 ; FIX(0.08131)
F_0_114 equ 7471 ; FIX(0.11400)
F_0_168 equ 11059 ; FIX(0.16874)
F_0_250 equ 16384 ; FIX(0.25000)
F_0_299 equ 19595 ; FIX(0.29900)
F_0_331 equ 21709 ; FIX(0.33126)
F_0_418 equ 27439 ; FIX(0.41869)
F_0_587 equ 38470 ; FIX(0.58700)
F_0_337 equ (F_0_587 - F_0_250) ; FIX(0.58700) - FIX(0.25000)
; --------------------------------------------------------------------------
SECTION SEG_CONST
ALIGNZ 32
GLOBAL_DATA(jconst_rgb_ycc_convert_sse2)
EXTN(jconst_rgb_ycc_convert_sse2):
PW_F0299_F0337 times 4 dw F_0_299, F_0_337
PW_F0114_F0250 times 4 dw F_0_114, F_0_250
PW_MF016_MF033 times 4 dw -F_0_168, -F_0_331
PW_MF008_MF041 times 4 dw -F_0_081, -F_0_418
PD_ONEHALFM1_CJ times 4 dd (1 << (SCALEBITS - 1)) - 1 + \
(CENTERJSAMPLE << SCALEBITS)
PD_ONEHALF times 4 dd (1 << (SCALEBITS - 1))
ALIGNZ 32
; --------------------------------------------------------------------------
SECTION SEG_TEXT
BITS 64
%include "jccolext-sse2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_RGB_RED
%define RGB_GREEN EXT_RGB_GREEN
%define RGB_BLUE EXT_RGB_BLUE
%define RGB_PIXELSIZE EXT_RGB_PIXELSIZE
%define jsimd_rgb_ycc_convert_sse2 jsimd_extrgb_ycc_convert_sse2
%include "jccolext-sse2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_RGBX_RED
%define RGB_GREEN EXT_RGBX_GREEN
%define RGB_BLUE EXT_RGBX_BLUE
%define RGB_PIXELSIZE EXT_RGBX_PIXELSIZE
%define jsimd_rgb_ycc_convert_sse2 jsimd_extrgbx_ycc_convert_sse2
%include "jccolext-sse2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_BGR_RED
%define RGB_GREEN EXT_BGR_GREEN
%define RGB_BLUE EXT_BGR_BLUE
%define RGB_PIXELSIZE EXT_BGR_PIXELSIZE
%define jsimd_rgb_ycc_convert_sse2 jsimd_extbgr_ycc_convert_sse2
%include "jccolext-sse2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_BGRX_RED
%define RGB_GREEN EXT_BGRX_GREEN
%define RGB_BLUE EXT_BGRX_BLUE
%define RGB_PIXELSIZE EXT_BGRX_PIXELSIZE
%define jsimd_rgb_ycc_convert_sse2 jsimd_extbgrx_ycc_convert_sse2
%include "jccolext-sse2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_XBGR_RED
%define RGB_GREEN EXT_XBGR_GREEN
%define RGB_BLUE EXT_XBGR_BLUE
%define RGB_PIXELSIZE EXT_XBGR_PIXELSIZE
%define jsimd_rgb_ycc_convert_sse2 jsimd_extxbgr_ycc_convert_sse2
%include "jccolext-sse2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_XRGB_RED
%define RGB_GREEN EXT_XRGB_GREEN
%define RGB_BLUE EXT_XRGB_BLUE
%define RGB_PIXELSIZE EXT_XRGB_PIXELSIZE
%define jsimd_rgb_ycc_convert_sse2 jsimd_extxrgb_ycc_convert_sse2
%include "jccolext-sse2.asm"
lib/libjpeg-turbo/simd/x86_64/jcgray-avx2.asm
Executable
+109
View File
@@ -0,0 +1,109 @@
;
; Grayscale colorspace conversion (64-bit AVX2)
;
; Copyright (C) 2011, 2016, 2024, D. R. Commander.
; Copyright (C) 2015, Intel Corporation.
;
; Based on the x86 SIMD extension for IJG JPEG library
; Copyright (C) 1999-2006, MIYASAKA Masaru.
; For conditions of distribution and use, see copyright notice in jsimdext.inc
;
; This file should be assembled with NASM (Netwide Assembler) or Yasm.
%include "jsimdext.inc"
; --------------------------------------------------------------------------
%define SCALEBITS 16
F_0_114 equ 7471 ; FIX(0.11400)
F_0_250 equ 16384 ; FIX(0.25000)
F_0_299 equ 19595 ; FIX(0.29900)
F_0_587 equ 38470 ; FIX(0.58700)
F_0_337 equ (F_0_587 - F_0_250) ; FIX(0.58700) - FIX(0.25000)
; --------------------------------------------------------------------------
SECTION SEG_CONST
ALIGNZ 32
GLOBAL_DATA(jconst_rgb_gray_convert_avx2)
EXTN(jconst_rgb_gray_convert_avx2):
PW_F0299_F0337 times 8 dw F_0_299, F_0_337
PW_F0114_F0250 times 8 dw F_0_114, F_0_250
PD_ONEHALF times 8 dd (1 << (SCALEBITS - 1))
ALIGNZ 32
; --------------------------------------------------------------------------
SECTION SEG_TEXT
BITS 64
%include "jcgryext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_RGB_RED
%define RGB_GREEN EXT_RGB_GREEN
%define RGB_BLUE EXT_RGB_BLUE
%define RGB_PIXELSIZE EXT_RGB_PIXELSIZE
%define jsimd_rgb_gray_convert_avx2 jsimd_extrgb_gray_convert_avx2
%include "jcgryext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_RGBX_RED
%define RGB_GREEN EXT_RGBX_GREEN
%define RGB_BLUE EXT_RGBX_BLUE
%define RGB_PIXELSIZE EXT_RGBX_PIXELSIZE
%define jsimd_rgb_gray_convert_avx2 jsimd_extrgbx_gray_convert_avx2
%include "jcgryext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_BGR_RED
%define RGB_GREEN EXT_BGR_GREEN
%define RGB_BLUE EXT_BGR_BLUE
%define RGB_PIXELSIZE EXT_BGR_PIXELSIZE
%define jsimd_rgb_gray_convert_avx2 jsimd_extbgr_gray_convert_avx2
%include "jcgryext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_BGRX_RED
%define RGB_GREEN EXT_BGRX_GREEN
%define RGB_BLUE EXT_BGRX_BLUE
%define RGB_PIXELSIZE EXT_BGRX_PIXELSIZE
%define jsimd_rgb_gray_convert_avx2 jsimd_extbgrx_gray_convert_avx2
%include "jcgryext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_XBGR_RED
%define RGB_GREEN EXT_XBGR_GREEN
%define RGB_BLUE EXT_XBGR_BLUE
%define RGB_PIXELSIZE EXT_XBGR_PIXELSIZE
%define jsimd_rgb_gray_convert_avx2 jsimd_extxbgr_gray_convert_avx2
%include "jcgryext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_XRGB_RED
%define RGB_GREEN EXT_XRGB_GREEN
%define RGB_BLUE EXT_XRGB_BLUE
%define RGB_PIXELSIZE EXT_XRGB_PIXELSIZE
%define jsimd_rgb_gray_convert_avx2 jsimd_extxrgb_gray_convert_avx2
%include "jcgryext-avx2.asm"
lib/libjpeg-turbo/simd/x86_64/jcgryext-sse2.asm
Executable
+411
View File
@@ -0,0 +1,411 @@
;
; Grayscale colorspace conversion (64-bit SSE2)
;
; Copyright (C) 2011, 2016, 2024-2025, D. R. Commander.
; Copyright (C) 2018, Matthias Räncker.
; Copyright (C) 2023, Aliaksiej Kandracienka.
;
; Based on the x86 SIMD extension for IJG JPEG library
; Copyright (C) 1999-2006, MIYASAKA Masaru.
; For conditions of distribution and use, see copyright notice in jsimdext.inc
;
; This file should be assembled with NASM (Netwide Assembler) or Yasm.
%include "jcolsamp.inc"
; --------------------------------------------------------------------------
;
; Convert some rows of samples to the JPEG colorspace.
;
; GLOBAL(void)
; jsimd_rgb_gray_convert_sse2(JDIMENSION img_width, JSAMPARRAY input_buf,
; JSAMPIMAGE output_buf, JDIMENSION output_row,
; int num_rows)
;
; r10d = JDIMENSION img_width
; r11 = JSAMPARRAY input_buf
; r12 = JSAMPIMAGE output_buf
; r13d = JDIMENSION output_row
; r14d = int num_rows
%define wk(i) r15 - (WK_NUM - (i)) * SIZEOF_XMMWORD ; xmmword wk[WK_NUM]
%define WK_NUM 2
align 32
GLOBAL_FUNCTION(jsimd_rgb_gray_convert_sse2)
EXTN(jsimd_rgb_gray_convert_sse2):
ENDBR64
push rbp
mov rbp, rsp
push r15
and rsp, byte (-SIZEOF_XMMWORD) ; align to 128 bits
; Allocate stack space for wk array. r15 is used to access it.
mov r15, rsp
sub rsp, byte (SIZEOF_XMMWORD * WK_NUM)
COLLECT_ARGS 5
push rbx
mov ecx, r10d
test rcx, rcx
jz near .return
push rcx
mov rsi, r12
mov ecx, r13d
mov rdip, JSAMPARRAY [rsi + 0 * SIZEOF_JSAMPARRAY]
lea rdi, [rdi + rcx * SIZEOF_JSAMPROW]
pop rcx
mov rsi, r11
mov eax, r14d
test rax, rax
jle near .return
.rowloop:
push rdi
push rsi
push rcx ; col
mov rsip, JSAMPROW [rsi] ; inptr
mov rdip, JSAMPROW [rdi] ; outptr0
cmp rcx, byte SIZEOF_XMMWORD
jae near .columnloop
%if RGB_PIXELSIZE == 3 ; ---------------
.column_ld1:
push rax
push rdx
lea rcx, [rcx + rcx * 2] ; imul ecx, RGB_PIXELSIZE
test cl, SIZEOF_BYTE
jz short .column_ld2
sub rcx, byte SIZEOF_BYTE
movzx rax, byte [rsi + rcx]
.column_ld2:
test cl, SIZEOF_WORD
jz short .column_ld4
sub rcx, byte SIZEOF_WORD
movzx rdx, word [rsi + rcx]
shl rax, WORD_BIT
or rax, rdx
.column_ld4:
movd xmmA, eax
pop rdx
pop rax
test cl, SIZEOF_DWORD
jz short .column_ld8
sub rcx, byte SIZEOF_DWORD
movd xmmF, XMM_DWORD [rsi + rcx]
pslldq xmmA, SIZEOF_DWORD
por xmmA, xmmF
.column_ld8:
test cl, SIZEOF_MMWORD
jz short .column_ld16
sub rcx, byte SIZEOF_MMWORD
movq xmmB, XMM_MMWORD [rsi + rcx]
pslldq xmmA, SIZEOF_MMWORD
por xmmA, xmmB
.column_ld16:
test cl, SIZEOF_XMMWORD
jz short .column_ld32
movdqa xmmF, xmmA
movdqu xmmA, XMMWORD [rsi + 0 * SIZEOF_XMMWORD]
mov rcx, SIZEOF_XMMWORD
jmp short .rgb_gray_cnv
.column_ld32:
test cl, 2 * SIZEOF_XMMWORD
mov rcx, SIZEOF_XMMWORD
jz short .rgb_gray_cnv
movdqa xmmB, xmmA
movdqu xmmA, XMMWORD [rsi + 0 * SIZEOF_XMMWORD]
movdqu xmmF, XMMWORD [rsi + 1 * SIZEOF_XMMWORD]
jmp short .rgb_gray_cnv
.columnloop:
movdqu xmmA, XMMWORD [rsi + 0 * SIZEOF_XMMWORD]
movdqu xmmF, XMMWORD [rsi + 1 * SIZEOF_XMMWORD]
movdqu xmmB, XMMWORD [rsi + 2 * SIZEOF_XMMWORD]
.rgb_gray_cnv:
; NOTE: The values of RGB_RED, RGB_GREEN, and RGB_BLUE determine the
; mapping of components A, B, and C to red, green, and blue.
;
; xmmA = (A0 B0 C0 A1 B1 C1 A2 B2 C2 A3 B3 C3 A4 B4 C4 A5)
; xmmF = (B5 C5 A6 B6 C6 A7 B7 C7 A8 B8 C8 A9 B9 C9 Aa Ba)
; xmmB = (Ca Ab Bb Cb Ac Bc Cc Ad Bd Cd Ae Be Ce Af Bf Cf)
movdqa xmmG, xmmA
pslldq xmmA, 8
; xmmA = (-- -- -- -- -- -- -- -- A0 B0 C0 A1 B1 C1 A2 B2)
psrldq xmmG, 8
; xmmG = (C2 A3 B3 C3 A4 B4 C4 A5 -- -- -- -- -- -- -- --)
punpckhbw xmmA, xmmF
; xmmA = (A0 A8 B0 B8 C0 C8 A1 A9 B1 B9 C1 C9 A2 Aa B2 Ba)
pslldq xmmF, 8
; xmmF = (-- -- -- -- -- -- -- -- B5 C5 A6 B6 C6 A7 B7 C7)
punpcklbw xmmG, xmmB
; xmmG = (C2 Ca A3 Ab B3 Bb C3 Cb A4 Ac B4 Bc C4 Cc A5 Ad)
punpckhbw xmmF, xmmB
; xmmF = (B5 Bd C5 Cd A6 Ae B6 Be C6 Ce A7 Af B7 Bf C7 Cf)
movdqa xmmD, xmmA
pslldq xmmA, 8
; xmmA = (-- -- -- -- -- -- -- -- A0 A8 B0 B8 C0 C8 A1 A9)
psrldq xmmD, 8
; xmmD = (B1 B9 C1 C9 A2 Aa B2 Ba -- -- -- -- -- -- -- --)
punpckhbw xmmA, xmmG
; xmmA = (A0 A4 A8 Ac B0 B4 B8 Bc C0 C4 C8 Cc A1 A5 A9 Ad)
pslldq xmmG, 8
; xmmG = (-- -- -- -- -- -- -- -- C2 Ca A3 Ab B3 Bb C3 Cb)
punpcklbw xmmD, xmmF
; xmmD = (B1 B5 B9 Bd C1 C5 C9 Cd A2 A6 Aa Ae B2 B6 Ba Be)
punpckhbw xmmG, xmmF
; xmmG = (C2 C6 Ca Ce A3 A7 Ab Af B3 B7 Bb Bf C3 C7 Cb Cf)
movdqa xmmE, xmmA
pslldq xmmA, 8
; xmmA = (-- -- -- -- -- -- -- -- A0 A4 A8 Ac B0 B4 B8 Bc)
psrldq xmmE, 8
; xmmE = (C0 C4 C8 Cc A1 A5 A9 Ad -- -- -- -- -- -- -- --)
punpckhbw xmmA, xmmD
; xmmA = (A0 A2 A4 A6 A8 Aa Ac Ae B0 B2 B4 B6 B8 Ba Bc Be)
pslldq xmmD, 8
; xmmD = (-- -- -- -- -- -- -- -- B1 B5 B9 Bd C1 C5 C9 Cd)
punpcklbw xmmE, xmmG
; xmmE = (C0 C2 C4 C6 C8 Ca Cc Ce A1 A3 A5 A7 A9 Ab Ad Af)
punpckhbw xmmD, xmmG
; xmmD = (B1 B3 B5 B7 B9 Bb Bd Bf C1 C3 C5 C7 C9 Cb Cd Cf)
pxor xmmH, xmmH
movdqa xmmC, xmmA
punpcklbw xmmA, xmmH ; xmmA = (A0 A2 A4 A6 A8 Aa Ac Ae) = AE
punpckhbw xmmC, xmmH ; xmmC = (B0 B2 B4 B6 B8 Ba Bc Be) = BE
movdqa xmmB, xmmE
punpcklbw xmmE, xmmH ; xmmE = (C0 C2 C4 C6 C8 Ca Cc Ce) = CE
punpckhbw xmmB, xmmH ; xmmB = (A1 A3 A5 A7 A9 Ab Ad Af) = AO
movdqa xmmF, xmmD
punpcklbw xmmD, xmmH ; xmmD = (B1 B3 B5 B7 B9 Bb Bd Bf) = BO
punpckhbw xmmF, xmmH ; xmmF = (C1 C3 C5 C7 C9 Cb Cd Cf) = CO
%else ; RGB_PIXELSIZE == 4 ; -----------
.column_ld1:
test cl, SIZEOF_XMMWORD / 16
jz short .column_ld2
sub rcx, byte SIZEOF_XMMWORD / 16
movd xmmA, XMM_DWORD [rsi + rcx * RGB_PIXELSIZE]
.column_ld2:
test cl, SIZEOF_XMMWORD / 8
jz short .column_ld4
sub rcx, byte SIZEOF_XMMWORD / 8
movq xmmE, XMM_MMWORD [rsi + rcx * RGB_PIXELSIZE]
pslldq xmmA, SIZEOF_MMWORD
por xmmA, xmmE
.column_ld4:
test cl, SIZEOF_XMMWORD / 4
jz short .column_ld8
sub rcx, byte SIZEOF_XMMWORD / 4
movdqa xmmE, xmmA
movdqu xmmA, XMMWORD [rsi + rcx * RGB_PIXELSIZE]
.column_ld8:
test cl, SIZEOF_XMMWORD / 2
mov rcx, SIZEOF_XMMWORD
jz short .rgb_gray_cnv
movdqa xmmF, xmmA
movdqa xmmH, xmmE
movdqu xmmA, XMMWORD [rsi + 0 * SIZEOF_XMMWORD]
movdqu xmmE, XMMWORD [rsi + 1 * SIZEOF_XMMWORD]
jmp short .rgb_gray_cnv
.columnloop:
movdqu xmmA, XMMWORD [rsi + 0 * SIZEOF_XMMWORD]
movdqu xmmE, XMMWORD [rsi + 1 * SIZEOF_XMMWORD]
movdqu xmmF, XMMWORD [rsi + 2 * SIZEOF_XMMWORD]
movdqu xmmH, XMMWORD [rsi + 3 * SIZEOF_XMMWORD]
.rgb_gray_cnv:
; NOTE: The values of RGB_RED, RGB_GREEN, and RGB_BLUE determine the
; mapping of components A, B, C, and D to red, green, and blue.
;
; xmmA = (A0 B0 C0 D0 A1 B1 C1 D1 A2 B2 C2 D2 A3 B3 C3 D3)
; xmmE = (A4 B4 C4 D4 A5 B5 C5 D5 A6 B6 C6 D6 A7 B7 C7 D7)
; xmmF = (A8 B8 C8 D8 A9 B9 C9 D9 Aa Ba Ca Da Ab Bb Cb Db)
; xmmH = (Ac Bc Cc Dc Ad Bd Cd Dd Ae Be Ce De Af Bf Cf Df)
movdqa xmmD, xmmA
punpcklbw xmmA, xmmE
; xmmA = (A0 A4 B0 B4 C0 C4 D0 D4 A1 A5 B1 B5 C1 C5 D1 D5)
punpckhbw xmmD, xmmE
; xmmD = (A2 A6 B2 B6 C2 C6 D2 D6 A3 A7 B3 B7 C3 C7 D3 D7)
movdqa xmmC, xmmF
punpcklbw xmmF, xmmH
; xmmF = (A8 Ac B8 Bc C8 Cc D8 Dc A9 Ad B9 Bd C9 Cd D9 Dd)
punpckhbw xmmC, xmmH
; xmmC = (Aa Ae Ba Be Ca Ce Da De Ab Af Bb Bf Cb Cf Db Df)
movdqa xmmB, xmmA
punpcklwd xmmA, xmmF
; xmmA = (A0 A4 A8 Ac B0 B4 B8 Bc C0 C4 C8 Cc D0 D4 D8 Dc)
punpckhwd xmmB, xmmF
; xmmB = (A1 A5 A9 Ad B1 B5 B9 Bd C1 C5 C9 Cd D1 D5 D9 Dd)
movdqa xmmG, xmmD
punpcklwd xmmD, xmmC
; xmmD = (A2 A6 Aa Ae B2 B6 Ba Be C2 C6 Ca Ce D2 D6 Da De)
punpckhwd xmmG, xmmC
; xmmG = (A3 A7 Ab Af B3 B7 Bb Bf C3 C7 Cb Cf D3 D7 Db Df)
movdqa xmmE, xmmA
punpcklbw xmmA, xmmD
; xmmA = (A0 A2 A4 A6 A8 Aa Ac Ae B0 B2 B4 B6 B8 Ba Bc Be)
punpckhbw xmmE, xmmD
; xmmE = (C0 C2 C4 C6 C8 Ca Cc Ce D0 D2 D4 D6 D8 Da Dc De)
movdqa xmmH, xmmB
punpcklbw xmmB, xmmG
; xmmB = (A1 A3 A5 A7 A9 Ab Ad Af B1 B3 B5 B7 B9 Bb Bd Bf)
punpckhbw xmmH, xmmG
; xmmH = (C1 C3 C5 C7 C9 Cb Cd Cf D1 D3 D5 D7 D9 Db Dd Df)
pxor xmmF, xmmF
movdqa xmmC, xmmA
punpcklbw xmmA, xmmF ; xmmA = (A0 A2 A4 A6 A8 Aa Ac Ae) = AE
punpckhbw xmmC, xmmF ; xmmC = (B0 B2 B4 B6 B8 Ba Bc Be) = BE
movdqa xmmD, xmmB
punpcklbw xmmB, xmmF ; xmmB = (A1 A3 A5 A7 A9 Ab Ad Af) = AO
punpckhbw xmmD, xmmF ; xmmD = (B1 B3 B5 B7 B9 Bb Bd Bf) = BO
movdqa xmmG, xmmE
punpcklbw xmmE, xmmF ; xmmE = (C0 C2 C4 C6 C8 Ca Cc Ce) = CE
punpckhbw xmmG, xmmF ; xmmG = (D0 D2 D4 D6 D8 Da Dc De) = DE
punpcklbw xmmF, xmmH
punpckhbw xmmH, xmmH
psrlw xmmF, BYTE_BIT ; xmmF = (C1 C3 C5 C7 C9 Cb Cd Cf) = CO
psrlw xmmH, BYTE_BIT ; xmmH = (D1 D3 D5 D7 D9 Db Dd Df) = DO
%endif ; RGB_PIXELSIZE ; ---------------
; xmm0 = (R0 R2 R4 R6 R8 Ra Rc Re) = RE
; xmm2 = (G0 G2 G4 G6 G8 Ga Gc Ge) = GE
; xmm4 = (B0 B2 B4 B6 B8 Ba Bc Be) = BE
; xmm1 = (R1 R3 R5 R7 R9 Rb Rd Rf) = RO
; xmm3 = (G1 G3 G5 G7 G9 Gb Gd Gf) = GO
; xmm5 = (B1 B3 B5 B7 B9 Bb Bd Bf) = BO
;
; (Original)
; Y = 0.29900 * R + 0.58700 * G + 0.11400 * B
;
; (This implementation)
; Y = 0.29900 * R + 0.33700 * G + 0.11400 * B + 0.25000 * G
movdqa xmm6, xmm1
punpcklwd xmm1, xmm3
punpckhwd xmm6, xmm3
pmaddwd xmm1, [rel PW_F0299_F0337]
; xmm1 = ROL * FIX(0.299) + GOL * FIX(0.337)
pmaddwd xmm6, [rel PW_F0299_F0337]
; xmm6 = ROH * FIX(0.299) + GOH * FIX(0.337)
movdqa xmm7, xmm6
; xmm7 = ROH * FIX(0.299) + GOH * FIX(0.337)
movdqa xmm6, xmm0
punpcklwd xmm0, xmm2
punpckhwd xmm6, xmm2
pmaddwd xmm0, [rel PW_F0299_F0337]
; xmm0 = REL * FIX(0.299) + GEL * FIX(0.337)
pmaddwd xmm6, [rel PW_F0299_F0337]
; xmm6 = REH * FIX(0.299) + GEH * FIX(0.337)
movdqa XMMWORD [wk(0)], xmm0
; wk(0) = REL * FIX(0.299) + GEL * FIX(0.337)
movdqa XMMWORD [wk(1)], xmm6
; wk(1) = REH * FIX(0.299) + GEH * FIX(0.337)
movdqa xmm0, xmm5 ; xmm0 = BO
movdqa xmm6, xmm4 ; xmm6 = BE
movdqa xmm4, xmm0
punpcklwd xmm0, xmm3
punpckhwd xmm4, xmm3
pmaddwd xmm0, [rel PW_F0114_F0250]
; xmm0 = BOL * FIX(0.114) + GOL * FIX(0.250)
pmaddwd xmm4, [rel PW_F0114_F0250]
; xmm4 = BOH * FIX(0.114) + GOH * FIX(0.250)
movdqa xmm3, [rel PD_ONEHALF] ; xmm3 = [PD_ONEHALF]
paddd xmm0, xmm1
paddd xmm4, xmm7
paddd xmm0, xmm3
paddd xmm4, xmm3
psrld xmm0, SCALEBITS ; xmm0 = YOL
psrld xmm4, SCALEBITS ; xmm4 = YOH
packssdw xmm0, xmm4 ; xmm0 = YO
movdqa xmm4, xmm6
punpcklwd xmm6, xmm2
punpckhwd xmm4, xmm2
pmaddwd xmm6, [rel PW_F0114_F0250]
; xmm6 = BEL * FIX(0.114) + GEL * FIX(0.250)
pmaddwd xmm4, [rel PW_F0114_F0250]
; xmm4 = BEH * FIX(0.114) + GEH * FIX(0.250)
movdqa xmm2, [rel PD_ONEHALF] ; xmm2 = [PD_ONEHALF]
paddd xmm6, XMMWORD [wk(0)]
paddd xmm4, XMMWORD [wk(1)]
paddd xmm6, xmm2
paddd xmm4, xmm2
psrld xmm6, SCALEBITS ; xmm6 = YEL
psrld xmm4, SCALEBITS ; xmm4 = YEH
packssdw xmm6, xmm4 ; xmm6 = YE
psllw xmm0, BYTE_BIT
por xmm6, xmm0 ; xmm6 = Y
movdqa XMMWORD [rdi], xmm6 ; Save Y
sub rcx, byte SIZEOF_XMMWORD
add rsi, byte RGB_PIXELSIZE * SIZEOF_XMMWORD ; inptr
add rdi, byte SIZEOF_XMMWORD ; outptr0
cmp rcx, byte SIZEOF_XMMWORD
jae near .columnloop
test rcx, rcx
jnz near .column_ld1
pop rcx ; col
pop rsi
pop rdi
add rsi, byte SIZEOF_JSAMPROW ; input_buf
add rdi, byte SIZEOF_JSAMPROW
dec rax ; num_rows
jg near .rowloop
.return:
pop rbx
UNCOLLECT_ARGS 5
lea rsp, [rbp - 8]
pop r15
pop rbp
ret
; For some reason, the OS X linker does not honor the request to align the
; segment unless we do this.
align 32
lib/libjpeg-turbo/simd/x86_64/jcphuff-sse2.asm
Executable
+626
View File
@@ -0,0 +1,626 @@
;
; Prepare data for progressive Huffman encoding (64-bit SSE2)
;
; Copyright (C) 2016, 2018, Matthieu Darbois.
; Copyright (C) 2023, Aliaksiej Kandracienka.
; Copyright (C) 2024-2025, D. R. Commander.
;
; Based on the x86 SIMD extension for IJG JPEG library
; Copyright (C) 1999-2006, MIYASAKA Masaru.
; For conditions of distribution and use, see copyright notice in jsimdext.inc
;
; This file should be assembled with NASM (Netwide Assembler) or Yasm.
;
; This file contains an SSE2 implementation of data preparation for progressive
; Huffman encoding. See jcphuff.c for more details.
%include "jsimdext.inc"
; --------------------------------------------------------------------------
SECTION SEG_TEXT
BITS 64
; --------------------------------------------------------------------------
; Macros to load data for jsimd_encode_mcu_AC_first_prepare_sse2() and
; jsimd_encode_mcu_AC_refine_prepare_sse2()
%macro LOAD16 0
pxor N0, N0
pxor N1, N1
mov T0d, INT [LUT + 0 * SIZEOF_INT]
mov T1d, INT [LUT + 8 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 0
pinsrw X1, word [BLOCK + T1 * 2], 0
mov T0d, INT [LUT + 1 * SIZEOF_INT]
mov T1d, INT [LUT + 9 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 1
pinsrw X1, word [BLOCK + T1 * 2], 1
mov T0d, INT [LUT + 2 * SIZEOF_INT]
mov T1d, INT [LUT + 10 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 2
pinsrw X1, word [BLOCK + T1 * 2], 2
mov T0d, INT [LUT + 3 * SIZEOF_INT]
mov T1d, INT [LUT + 11 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 3
pinsrw X1, word [BLOCK + T1 * 2], 3
mov T0d, INT [LUT + 4 * SIZEOF_INT]
mov T1d, INT [LUT + 12 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 4
pinsrw X1, word [BLOCK + T1 * 2], 4
mov T0d, INT [LUT + 5 * SIZEOF_INT]
mov T1d, INT [LUT + 13 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 5
pinsrw X1, word [BLOCK + T1 * 2], 5
mov T0d, INT [LUT + 6 * SIZEOF_INT]
mov T1d, INT [LUT + 14 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 6
pinsrw X1, word [BLOCK + T1 * 2], 6
mov T0d, INT [LUT + 7 * SIZEOF_INT]
mov T1d, INT [LUT + 15 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 7
pinsrw X1, word [BLOCK + T1 * 2], 7
%endmacro
%macro LOAD15 0
pxor N0, N0
pxor N1, N1
pxor X1, X1
mov T0d, INT [LUT + 0 * SIZEOF_INT]
mov T1d, INT [LUT + 8 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 0
pinsrw X1, word [BLOCK + T1 * 2], 0
mov T0d, INT [LUT + 1 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 1
mov T0d, INT [LUT + 2 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 2
mov T0d, INT [LUT + 3 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 3
mov T0d, INT [LUT + 4 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 4
mov T0d, INT [LUT + 5 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 5
mov T0d, INT [LUT + 6 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 6
mov T0d, INT [LUT + 7 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 7
cmp LENEND, 2
jl %%.ELOAD15
mov T1d, INT [LUT + 9 * SIZEOF_INT]
pinsrw X1, word [BLOCK + T1 * 2], 1
cmp LENEND, 3
jl %%.ELOAD15
mov T1d, INT [LUT + 10 * SIZEOF_INT]
pinsrw X1, word [BLOCK + T1 * 2], 2
cmp LENEND, 4
jl %%.ELOAD15
mov T1d, INT [LUT + 11 * SIZEOF_INT]
pinsrw X1, word [BLOCK + T1 * 2], 3
cmp LENEND, 5
jl %%.ELOAD15
mov T1d, INT [LUT + 12 * SIZEOF_INT]
pinsrw X1, word [BLOCK + T1 * 2], 4
cmp LENEND, 6
jl %%.ELOAD15
mov T1d, INT [LUT + 13 * SIZEOF_INT]
pinsrw X1, word [BLOCK + T1 * 2], 5
cmp LENEND, 7
jl %%.ELOAD15
mov T1d, INT [LUT + 14 * SIZEOF_INT]
pinsrw X1, word [BLOCK + T1 * 2], 6
%%.ELOAD15:
%endmacro
%macro LOAD8 0
pxor N0, N0
mov T0d, INT [LUT + 0 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 0
mov T0d, INT [LUT + 1 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 1
mov T0d, INT [LUT + 2 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 2
mov T0d, INT [LUT + 3 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 3
mov T0d, INT [LUT + 4 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 4
mov T0d, INT [LUT + 5 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 5
mov T0d, INT [LUT + 6 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 6
mov T0d, INT [LUT + 7 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T0 * 2], 7
%endmacro
%macro LOAD7 0
pxor N0, N0
pxor X0, X0
mov T1d, INT [LUT + 0 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T1 * 2], 0
cmp LENEND, 2
jl %%.ELOAD7
mov T1d, INT [LUT + 1 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T1 * 2], 1
cmp LENEND, 3
jl %%.ELOAD7
mov T1d, INT [LUT + 2 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T1 * 2], 2
cmp LENEND, 4
jl %%.ELOAD7
mov T1d, INT [LUT + 3 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T1 * 2], 3
cmp LENEND, 5
jl %%.ELOAD7
mov T1d, INT [LUT + 4 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T1 * 2], 4
cmp LENEND, 6
jl %%.ELOAD7
mov T1d, INT [LUT + 5 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T1 * 2], 5
cmp LENEND, 7
jl %%.ELOAD7
mov T1d, INT [LUT + 6 * SIZEOF_INT]
pinsrw X0, word [BLOCK + T1 * 2], 6
%%.ELOAD7:
%endmacro
%macro REDUCE0 0
movdqa xmm0, XMMWORD [VALUES + ( 0 * 2)]
movdqa xmm1, XMMWORD [VALUES + ( 8 * 2)]
movdqa xmm2, XMMWORD [VALUES + (16 * 2)]
movdqa xmm3, XMMWORD [VALUES + (24 * 2)]
movdqa xmm4, XMMWORD [VALUES + (32 * 2)]
movdqa xmm5, XMMWORD [VALUES + (40 * 2)]
movdqa xmm6, XMMWORD [VALUES + (48 * 2)]
movdqa xmm7, XMMWORD [VALUES + (56 * 2)]
pcmpeqw xmm0, ZERO
pcmpeqw xmm1, ZERO
pcmpeqw xmm2, ZERO
pcmpeqw xmm3, ZERO
pcmpeqw xmm4, ZERO
pcmpeqw xmm5, ZERO
pcmpeqw xmm6, ZERO
pcmpeqw xmm7, ZERO
packsswb xmm0, xmm1
packsswb xmm2, xmm3
packsswb xmm4, xmm5
packsswb xmm6, xmm7
pmovmskb eax, xmm0
pmovmskb ecx, xmm2
pmovmskb edx, xmm4
pmovmskb esi, xmm6
shl rcx, 16
shl rdx, 32
shl rsi, 48
or rax, rcx
or rdx, rsi
or rax, rdx
not rax
mov MMWORD [r15], rax
%endmacro
; Prepare data for jsimd_encode_mcu_AC_first().
;
; GLOBAL(void)
; jsimd_encode_mcu_AC_first_prepare_sse2(const JCOEF *block,
; const int *jpeg_natural_order_start,
; int Sl, int Al, JCOEF *values,
; size_t *zerobits)
;
; r10 = const JCOEF *block
; r11 = const int *jpeg_natural_order_start
; r12 = int Sl
; r13 = int Al
; r14 = JCOEF *values
; r15 = size_t *zerobits
%define ZERO xmm9
%define X0 xmm0
%define X1 xmm1
%define N0 xmm2
%define N1 xmm3
%define AL xmm4
%define K eax
%define LUT r11
%define T0 rcx
%define T0d ecx
%define T1 rdx
%define T1d edx
%define BLOCK r10
%define VALUES r14
%define LEN r12d
%define LENEND r13d
align 32
GLOBAL_FUNCTION(jsimd_encode_mcu_AC_first_prepare_sse2)
EXTN(jsimd_encode_mcu_AC_first_prepare_sse2):
ENDBR64
push rbp
mov rbp, rsp
and rsp, byte (-SIZEOF_XMMWORD) ; align to 128 bits
sub rsp, SIZEOF_XMMWORD
movdqa XMMWORD [rsp], ZERO
COLLECT_ARGS 6
movd AL, r13d
pxor ZERO, ZERO
mov K, LEN
mov LENEND, LEN
and K, -16
and LENEND, 7
shr K, 4
jz .ELOOP16
.BLOOP16:
LOAD16
pcmpgtw N0, X0
pcmpgtw N1, X1
paddw X0, N0
paddw X1, N1
pxor X0, N0
pxor X1, N1
psrlw X0, AL
psrlw X1, AL
pxor N0, X0
pxor N1, X1
movdqa XMMWORD [VALUES + (0) * 2], X0
movdqa XMMWORD [VALUES + (8) * 2], X1
movdqa XMMWORD [VALUES + (0 + DCTSIZE2) * 2], N0
movdqa XMMWORD [VALUES + (8 + DCTSIZE2) * 2], N1
add VALUES, 16 * 2
add LUT, 16 * SIZEOF_INT
dec K
jnz .BLOOP16
test LEN, 15
je .PADDING
.ELOOP16:
test LEN, 8
jz .TRY7
test LEN, 7
jz .TRY8
LOAD15
pcmpgtw N0, X0
pcmpgtw N1, X1
paddw X0, N0
paddw X1, N1
pxor X0, N0
pxor X1, N1
psrlw X0, AL
psrlw X1, AL
pxor N0, X0
pxor N1, X1
movdqa XMMWORD [VALUES + (0) * 2], X0
movdqa XMMWORD [VALUES + (8) * 2], X1
movdqa XMMWORD [VALUES + (0 + DCTSIZE2) * 2], N0
movdqa XMMWORD [VALUES + (8 + DCTSIZE2) * 2], N1
add VALUES, 16 * 2
jmp .PADDING
.TRY8:
LOAD8
pcmpgtw N0, X0
paddw X0, N0
pxor X0, N0
psrlw X0, AL
pxor N0, X0
movdqa XMMWORD [VALUES + (0) * 2], X0
movdqa XMMWORD [VALUES + (0 + DCTSIZE2) * 2], N0
add VALUES, 8 * 2
jmp .PADDING
.TRY7:
LOAD7
pcmpgtw N0, X0
paddw X0, N0
pxor X0, N0
psrlw X0, AL
pxor N0, X0
movdqa XMMWORD [VALUES + (0) * 2], X0
movdqa XMMWORD [VALUES + (0 + DCTSIZE2) * 2], N0
add VALUES, 8 * 2
.PADDING:
mov K, LEN
add K, 7
and K, -8
shr K, 3
sub K, DCTSIZE2 / 8
jz .EPADDING
align 16
.ZEROLOOP:
movdqa XMMWORD [VALUES + 0], ZERO
add VALUES, 8 * 2
inc K
jnz .ZEROLOOP
.EPADDING:
sub VALUES, DCTSIZE2 * 2
REDUCE0
UNCOLLECT_ARGS 6
movdqa ZERO, XMMWORD [rsp]
mov rsp, rbp
pop rbp
ret
%undef ZERO
%undef X0
%undef X1
%undef N0
%undef N1
%undef AL
%undef K
%undef LUT
%undef T0
%undef T0d
%undef T1
%undef T1d
%undef BLOCK
%undef VALUES
%undef LEN
%undef LENEND
; Prepare data for jsimd_encode_mcu_AC_refine().
;
; GLOBAL(int)
; jsimd_encode_mcu_AC_refine_prepare_sse2(const JCOEF *block,
; const int *jpeg_natural_order_start,
; int Sl, int Al, JCOEF *absvalues,
; size_t *bits)
;
; r10 = const JCOEF *block
; r11 = const int *jpeg_natural_order_start
; r12 = int Sl
; r13 = int Al
; r14 = JCOEF *values
; r15 = size_t *bits
%define ZERO xmm9
%define ONE xmm5
%define X0 xmm0
%define X1 xmm1
%define N0 xmm2
%define N1 xmm3
%define AL xmm4
%define K eax
%define KK r9d
%define EOB r8d
%define SIGN rdi
%define LUT r11
%define T0 rcx
%define T0d ecx
%define T1 rdx
%define T1d edx
%define BLOCK r10
%define VALUES r14
%define LEN r12d
%define LENEND r13d
align 32
GLOBAL_FUNCTION(jsimd_encode_mcu_AC_refine_prepare_sse2)
EXTN(jsimd_encode_mcu_AC_refine_prepare_sse2):
ENDBR64
push rbp
mov rbp, rsp
and rsp, byte (-SIZEOF_XMMWORD) ; align to 128 bits
sub rsp, SIZEOF_XMMWORD
movdqa XMMWORD [rsp], ZERO
COLLECT_ARGS 6
xor SIGN, SIGN
xor EOB, EOB
xor KK, KK
movd AL, r13d
pxor ZERO, ZERO
pcmpeqw ONE, ONE
psrlw ONE, 15
mov K, LEN
mov LENEND, LEN
and K, -16
and LENEND, 7
shr K, 4
jz .ELOOPR16
.BLOOPR16:
LOAD16
pcmpgtw N0, X0
pcmpgtw N1, X1
paddw X0, N0
paddw X1, N1
pxor X0, N0
pxor X1, N1
psrlw X0, AL
psrlw X1, AL
movdqa XMMWORD [VALUES + (0) * 2], X0
movdqa XMMWORD [VALUES + (8) * 2], X1
pcmpeqw X0, ONE
pcmpeqw X1, ONE
packsswb N0, N1
packsswb X0, X1
pmovmskb T0d, N0 ; lsignbits.val16u[k >> 4] = _mm_movemask_epi8(neg);
pmovmskb T1d, X0 ; idx = _mm_movemask_epi8(x1);
shr SIGN, 16 ; make room for sizebits
shl T0, 48
or SIGN, T0
bsr T1d, T1d ; idx = 16 - (__builtin_clz(idx) >> 1);
jz .CONTINUER16 ; if (idx) {
mov EOB, KK
add EOB, T1d ; EOB = k + idx;
.CONTINUER16:
add VALUES, 16 * 2
add LUT, 16 * SIZEOF_INT
add KK, 16
dec K
jnz .BLOOPR16
test LEN, 15
je .PADDINGR
.ELOOPR16:
test LEN, 8
jz .TRYR7
test LEN, 7
jz .TRYR8
LOAD15
pcmpgtw N0, X0
pcmpgtw N1, X1
paddw X0, N0
paddw X1, N1
pxor X0, N0
pxor X1, N1
psrlw X0, AL
psrlw X1, AL
movdqa XMMWORD [VALUES + (0) * 2], X0
movdqa XMMWORD [VALUES + (8) * 2], X1
pcmpeqw X0, ONE
pcmpeqw X1, ONE
packsswb N0, N1
packsswb X0, X1
pmovmskb T0d, N0 ; lsignbits.val16u[k >> 4] = _mm_movemask_epi8(neg);
pmovmskb T1d, X0 ; idx = _mm_movemask_epi8(x1);
shr SIGN, 16 ; make room for sizebits
shl T0, 48
or SIGN, T0
bsr T1d, T1d ; idx = 16 - (__builtin_clz(idx) >> 1);
jz .CONTINUER15 ; if (idx) {
mov EOB, KK
add EOB, T1d ; EOB = k + idx;
.CONTINUER15:
add VALUES, 16 * 2
jmp .PADDINGR
.TRYR8:
LOAD8
pcmpgtw N0, X0
paddw X0, N0
pxor X0, N0
psrlw X0, AL
movdqa XMMWORD [VALUES + (0) * 2], X0
pcmpeqw X0, ONE
packsswb N0, ZERO
packsswb X0, ZERO
pmovmskb T0d, N0 ; lsignbits.val16u[k >> 4] = _mm_movemask_epi8(neg);
pmovmskb T1d, X0 ; idx = _mm_movemask_epi8(x1);
shr SIGN, 8 ; make room for sizebits
shl T0, 56
or SIGN, T0
bsr T1d, T1d ; idx = 16 - (__builtin_clz(idx) >> 1);
jz .CONTINUER8 ; if (idx) {
mov EOB, KK
add EOB, T1d ; EOB = k + idx;
.CONTINUER8:
add VALUES, 8 * 2
jmp .PADDINGR
.TRYR7:
LOAD7
pcmpgtw N0, X0
paddw X0, N0
pxor X0, N0
psrlw X0, AL
movdqa XMMWORD [VALUES + (0) * 2], X0
pcmpeqw X0, ONE
packsswb N0, ZERO
packsswb X0, ZERO
pmovmskb T0d, N0 ; lsignbits.val16u[k >> 4] = _mm_movemask_epi8(neg);
pmovmskb T1d, X0 ; idx = _mm_movemask_epi8(x1);
shr SIGN, 8 ; make room for sizebits
shl T0, 56
or SIGN, T0
bsr T1d, T1d ; idx = 16 - (__builtin_clz(idx) >> 1);
jz .CONTINUER7 ; if (idx) {
mov EOB, KK
add EOB, T1d ; EOB = k + idx;
.CONTINUER7:
add VALUES, 8 * 2
.PADDINGR:
mov K, LEN
add K, 7
and K, -8
shr K, 3
sub K, DCTSIZE2 / 8
jz .EPADDINGR
align 16
.ZEROLOOPR:
movdqa XMMWORD [VALUES + 0], ZERO
shr SIGN, 8
add VALUES, 8 * 2
inc K
jnz .ZEROLOOPR
.EPADDINGR:
not SIGN
sub VALUES, DCTSIZE2 * 2
mov MMWORD [r15 + SIZEOF_MMWORD], SIGN
REDUCE0
mov eax, EOB
UNCOLLECT_ARGS 6
movdqa ZERO, XMMWORD [rsp]
mov rsp, rbp
pop rbp
ret
%undef ZERO
%undef ONE
%undef X0
%undef X1
%undef N0
%undef N1
%undef AL
%undef K
%undef KK
%undef EOB
%undef SIGN
%undef LUT
%undef T0
%undef T0d
%undef T1
%undef T1d
%undef BLOCK
%undef VALUES
%undef LEN
%undef LENEND
; For some reason, the OS X linker does not honor the request to align the
; segment unless we do this.
align 32
lib/libjpeg-turbo/simd/x86_64/jdcolor-avx2.asm
Executable
+114
View File
@@ -0,0 +1,114 @@
;
; Colorspace conversion (64-bit AVX2)
;
; Copyright 2009 Pierre Ossman <[email protected]> for Cendio AB
; Copyright (C) 2009, 2016, 2024, D. R. Commander.
; Copyright (C) 2015, Intel Corporation.
;
; Based on the x86 SIMD extension for IJG JPEG library
; Copyright (C) 1999-2006, MIYASAKA Masaru.
; For conditions of distribution and use, see copyright notice in jsimdext.inc
;
; This file should be assembled with NASM (Netwide Assembler) or Yasm.
%include "jsimdext.inc"
; --------------------------------------------------------------------------
%define SCALEBITS 16
F_0_344 equ 22554 ; FIX(0.34414)
F_0_714 equ 46802 ; FIX(0.71414)
F_1_402 equ 91881 ; FIX(1.40200)
F_1_772 equ 116130 ; FIX(1.77200)
F_0_402 equ (F_1_402 - 65536) ; FIX(1.40200) - FIX(1)
F_0_285 equ ( 65536 - F_0_714) ; FIX(1) - FIX(0.71414)
F_0_228 equ (131072 - F_1_772) ; FIX(2) - FIX(1.77200)
; --------------------------------------------------------------------------
SECTION SEG_CONST
ALIGNZ 32
GLOBAL_DATA(jconst_ycc_rgb_convert_avx2)
EXTN(jconst_ycc_rgb_convert_avx2):
PW_F0402 times 16 dw F_0_402
PW_MF0228 times 16 dw -F_0_228
PW_MF0344_F0285 times 8 dw -F_0_344, F_0_285
PW_ONE times 16 dw 1
PD_ONEHALF times 8 dd 1 << (SCALEBITS - 1)
ALIGNZ 32
; --------------------------------------------------------------------------
SECTION SEG_TEXT
BITS 64
%include "jdcolext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_RGB_RED
%define RGB_GREEN EXT_RGB_GREEN
%define RGB_BLUE EXT_RGB_BLUE
%define RGB_PIXELSIZE EXT_RGB_PIXELSIZE
%define jsimd_ycc_rgb_convert_avx2 jsimd_ycc_extrgb_convert_avx2
%include "jdcolext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_RGBX_RED
%define RGB_GREEN EXT_RGBX_GREEN
%define RGB_BLUE EXT_RGBX_BLUE
%define RGB_PIXELSIZE EXT_RGBX_PIXELSIZE
%define jsimd_ycc_rgb_convert_avx2 jsimd_ycc_extrgbx_convert_avx2
%include "jdcolext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_BGR_RED
%define RGB_GREEN EXT_BGR_GREEN
%define RGB_BLUE EXT_BGR_BLUE
%define RGB_PIXELSIZE EXT_BGR_PIXELSIZE
%define jsimd_ycc_rgb_convert_avx2 jsimd_ycc_extbgr_convert_avx2
%include "jdcolext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_BGRX_RED
%define RGB_GREEN EXT_BGRX_GREEN
%define RGB_BLUE EXT_BGRX_BLUE
%define RGB_PIXELSIZE EXT_BGRX_PIXELSIZE
%define jsimd_ycc_rgb_convert_avx2 jsimd_ycc_extbgrx_convert_avx2
%include "jdcolext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_XBGR_RED
%define RGB_GREEN EXT_XBGR_GREEN
%define RGB_BLUE EXT_XBGR_BLUE
%define RGB_PIXELSIZE EXT_XBGR_PIXELSIZE
%define jsimd_ycc_rgb_convert_avx2 jsimd_ycc_extxbgr_convert_avx2
%include "jdcolext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_XRGB_RED
%define RGB_GREEN EXT_XRGB_GREEN
%define RGB_BLUE EXT_XRGB_BLUE
%define RGB_PIXELSIZE EXT_XRGB_PIXELSIZE
%define jsimd_ycc_rgb_convert_avx2 jsimd_ycc_extxrgb_convert_avx2
%include "jdcolext-avx2.asm"
lib/libjpeg-turbo/simd/x86_64/jdcolor-sse2.asm
Executable
+113
View File
@@ -0,0 +1,113 @@
;
; Colorspace conversion (64-bit SSE2)
;
; Copyright 2009 Pierre Ossman <[email protected]> for Cendio AB
; Copyright (C) 2009, 2016, 2024, D. R. Commander.
;
; Based on the x86 SIMD extension for IJG JPEG library
; Copyright (C) 1999-2006, MIYASAKA Masaru.
; For conditions of distribution and use, see copyright notice in jsimdext.inc
;
; This file should be assembled with NASM (Netwide Assembler) or Yasm.
%include "jsimdext.inc"
; --------------------------------------------------------------------------
%define SCALEBITS 16
F_0_344 equ 22554 ; FIX(0.34414)
F_0_714 equ 46802 ; FIX(0.71414)
F_1_402 equ 91881 ; FIX(1.40200)
F_1_772 equ 116130 ; FIX(1.77200)
F_0_402 equ (F_1_402 - 65536) ; FIX(1.40200) - FIX(1)
F_0_285 equ ( 65536 - F_0_714) ; FIX(1) - FIX(0.71414)
F_0_228 equ (131072 - F_1_772) ; FIX(2) - FIX(1.77200)
; --------------------------------------------------------------------------
SECTION SEG_CONST
ALIGNZ 32
GLOBAL_DATA(jconst_ycc_rgb_convert_sse2)
EXTN(jconst_ycc_rgb_convert_sse2):
PW_F0402 times 8 dw F_0_402
PW_MF0228 times 8 dw -F_0_228
PW_MF0344_F0285 times 4 dw -F_0_344, F_0_285
PW_ONE times 8 dw 1
PD_ONEHALF times 4 dd 1 << (SCALEBITS - 1)
ALIGNZ 32
; --------------------------------------------------------------------------
SECTION SEG_TEXT
BITS 64
%include "jdcolext-sse2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_RGB_RED
%define RGB_GREEN EXT_RGB_GREEN
%define RGB_BLUE EXT_RGB_BLUE
%define RGB_PIXELSIZE EXT_RGB_PIXELSIZE
%define jsimd_ycc_rgb_convert_sse2 jsimd_ycc_extrgb_convert_sse2
%include "jdcolext-sse2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_RGBX_RED
%define RGB_GREEN EXT_RGBX_GREEN
%define RGB_BLUE EXT_RGBX_BLUE
%define RGB_PIXELSIZE EXT_RGBX_PIXELSIZE
%define jsimd_ycc_rgb_convert_sse2 jsimd_ycc_extrgbx_convert_sse2
%include "jdcolext-sse2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_BGR_RED
%define RGB_GREEN EXT_BGR_GREEN
%define RGB_BLUE EXT_BGR_BLUE
%define RGB_PIXELSIZE EXT_BGR_PIXELSIZE
%define jsimd_ycc_rgb_convert_sse2 jsimd_ycc_extbgr_convert_sse2
%include "jdcolext-sse2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_BGRX_RED
%define RGB_GREEN EXT_BGRX_GREEN
%define RGB_BLUE EXT_BGRX_BLUE
%define RGB_PIXELSIZE EXT_BGRX_PIXELSIZE
%define jsimd_ycc_rgb_convert_sse2 jsimd_ycc_extbgrx_convert_sse2
%include "jdcolext-sse2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_XBGR_RED
%define RGB_GREEN EXT_XBGR_GREEN
%define RGB_BLUE EXT_XBGR_BLUE
%define RGB_PIXELSIZE EXT_XBGR_PIXELSIZE
%define jsimd_ycc_rgb_convert_sse2 jsimd_ycc_extxbgr_convert_sse2
%include "jdcolext-sse2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_XRGB_RED
%define RGB_GREEN EXT_XRGB_GREEN
%define RGB_BLUE EXT_XRGB_BLUE
%define RGB_PIXELSIZE EXT_XRGB_PIXELSIZE
%define jsimd_ycc_rgb_convert_sse2 jsimd_ycc_extxrgb_convert_sse2
%include "jdcolext-sse2.asm"
lib/libjpeg-turbo/simd/x86_64/jdmerge-avx2.asm
Executable
+132
View File
@@ -0,0 +1,132 @@
;
; Merged upsampling/color conversion (64-bit AVX2)
;
; Copyright 2009 Pierre Ossman <[email protected]> for Cendio AB
; Copyright (C) 2009, 2016, 2024, D. R. Commander.
; Copyright (C) 2015, Intel Corporation.
;
; Based on the x86 SIMD extension for IJG JPEG library
; Copyright (C) 1999-2006, MIYASAKA Masaru.
; For conditions of distribution and use, see copyright notice in jsimdext.inc
;
; This file should be assembled with NASM (Netwide Assembler) or Yasm.
%include "jsimdext.inc"
; --------------------------------------------------------------------------
%define SCALEBITS 16
F_0_344 equ 22554 ; FIX(0.34414)
F_0_714 equ 46802 ; FIX(0.71414)
F_1_402 equ 91881 ; FIX(1.40200)
F_1_772 equ 116130 ; FIX(1.77200)
F_0_402 equ (F_1_402 - 65536) ; FIX(1.40200) - FIX(1)
F_0_285 equ ( 65536 - F_0_714) ; FIX(1) - FIX(0.71414)
F_0_228 equ (131072 - F_1_772) ; FIX(2) - FIX(1.77200)
; --------------------------------------------------------------------------
SECTION SEG_CONST
ALIGNZ 32
GLOBAL_DATA(jconst_merged_upsample_avx2)
EXTN(jconst_merged_upsample_avx2):
PW_F0402 times 16 dw F_0_402
PW_MF0228 times 16 dw -F_0_228
PW_MF0344_F0285 times 8 dw -F_0_344, F_0_285
PW_ONE times 16 dw 1
PD_ONEHALF times 8 dd 1 << (SCALEBITS - 1)
ALIGNZ 32
; --------------------------------------------------------------------------
SECTION SEG_TEXT
BITS 64
%include "jdmrgext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_RGB_RED
%define RGB_GREEN EXT_RGB_GREEN
%define RGB_BLUE EXT_RGB_BLUE
%define RGB_PIXELSIZE EXT_RGB_PIXELSIZE
%define jsimd_h2v1_merged_upsample_avx2 \
jsimd_h2v1_extrgb_merged_upsample_avx2
%define jsimd_h2v2_merged_upsample_avx2 \
jsimd_h2v2_extrgb_merged_upsample_avx2
%include "jdmrgext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_RGBX_RED
%define RGB_GREEN EXT_RGBX_GREEN
%define RGB_BLUE EXT_RGBX_BLUE
%define RGB_PIXELSIZE EXT_RGBX_PIXELSIZE
%define jsimd_h2v1_merged_upsample_avx2 \
jsimd_h2v1_extrgbx_merged_upsample_avx2
%define jsimd_h2v2_merged_upsample_avx2 \
jsimd_h2v2_extrgbx_merged_upsample_avx2
%include "jdmrgext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_BGR_RED
%define RGB_GREEN EXT_BGR_GREEN
%define RGB_BLUE EXT_BGR_BLUE
%define RGB_PIXELSIZE EXT_BGR_PIXELSIZE
%define jsimd_h2v1_merged_upsample_avx2 \
jsimd_h2v1_extbgr_merged_upsample_avx2
%define jsimd_h2v2_merged_upsample_avx2 \
jsimd_h2v2_extbgr_merged_upsample_avx2
%include "jdmrgext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_BGRX_RED
%define RGB_GREEN EXT_BGRX_GREEN
%define RGB_BLUE EXT_BGRX_BLUE
%define RGB_PIXELSIZE EXT_BGRX_PIXELSIZE
%define jsimd_h2v1_merged_upsample_avx2 \
jsimd_h2v1_extbgrx_merged_upsample_avx2
%define jsimd_h2v2_merged_upsample_avx2 \
jsimd_h2v2_extbgrx_merged_upsample_avx2
%include "jdmrgext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_XBGR_RED
%define RGB_GREEN EXT_XBGR_GREEN
%define RGB_BLUE EXT_XBGR_BLUE
%define RGB_PIXELSIZE EXT_XBGR_PIXELSIZE
%define jsimd_h2v1_merged_upsample_avx2 \
jsimd_h2v1_extxbgr_merged_upsample_avx2
%define jsimd_h2v2_merged_upsample_avx2 \
jsimd_h2v2_extxbgr_merged_upsample_avx2
%include "jdmrgext-avx2.asm"
%undef RGB_RED
%undef RGB_GREEN
%undef RGB_BLUE
%undef RGB_PIXELSIZE
%define RGB_RED EXT_XRGB_RED
%define RGB_GREEN EXT_XRGB_GREEN
%define RGB_BLUE EXT_XRGB_BLUE
%define RGB_PIXELSIZE EXT_XRGB_PIXELSIZE
%define jsimd_h2v1_merged_upsample_avx2 \
jsimd_h2v1_extxrgb_merged_upsample_avx2
%define jsimd_h2v2_merged_upsample_avx2 \
jsimd_h2v2_extxrgb_merged_upsample_avx2
%include "jdmrgext-avx2.asm"
lib/libjpeg-turbo/simd/x86_64/jdmrgext-avx2.asm
Executable
+658
View File
@@ -0,0 +1,658 @@
;
; Merged upsampling/color conversion (64-bit AVX2)
;
; Copyright 2009, 2012 Pierre Ossman <[email protected]> for Cendio AB
; Copyright (C) 2009, 2012, 2016, 2024-2025, D. R. Commander.
; Copyright (C) 2015, Intel Corporation.
; Copyright (C) 2018, Matthias Räncker.
; Copyright (C) 2023, Aliaksiej Kandracienka.
;
; Based on the x86 SIMD extension for IJG JPEG library
; Copyright (C) 1999-2006, MIYASAKA Masaru.
; For conditions of distribution and use, see copyright notice in jsimdext.inc
;
; This file should be assembled with NASM (Netwide Assembler) or Yasm.
%include "jcolsamp.inc"
; --------------------------------------------------------------------------
;
; Upsample and color convert for the case of 2:1 horizontal and 1:1 vertical.
;
; GLOBAL(void)
; jsimd_h2v1_merged_upsample_avx2(JDIMENSION output_width,
; JSAMPIMAGE input_buf,
; JDIMENSION in_row_group_ctr,
; JSAMPARRAY output_buf)
;
; r10d = JDIMENSION output_width
; r11 = JSAMPIMAGE input_buf
; r12d = JDIMENSION in_row_group_ctr
; r13 = JSAMPARRAY output_buf
%define wk(i) r15 - (WK_NUM - (i)) * SIZEOF_YMMWORD ; ymmword wk[WK_NUM]
%define WK_NUM 3
align 32
GLOBAL_FUNCTION(jsimd_h2v1_merged_upsample_avx2)
EXTN(jsimd_h2v1_merged_upsample_avx2):
ENDBR64
push rbp
mov rbp, rsp
push r15
and rsp, byte (-SIZEOF_YMMWORD) ; align to 256 bits
; Allocate stack space for wk array. r15 is used to access it.
mov r15, rsp
sub rsp, SIZEOF_YMMWORD * WK_NUM
COLLECT_ARGS 4
push rbx
mov ecx, r10d ; col
test rcx, rcx
jz near .return
push rcx
mov rdi, r11
mov ecx, r12d
mov rsip, JSAMPARRAY [rdi + 0 * SIZEOF_JSAMPARRAY]
mov rbxp, JSAMPARRAY [rdi + 1 * SIZEOF_JSAMPARRAY]
mov rdxp, JSAMPARRAY [rdi + 2 * SIZEOF_JSAMPARRAY]
mov rdi, r13
mov rsip, JSAMPROW [rsi + rcx * SIZEOF_JSAMPROW] ; inptr0
mov rbxp, JSAMPROW [rbx + rcx * SIZEOF_JSAMPROW] ; inptr1
mov rdxp, JSAMPROW [rdx + rcx * SIZEOF_JSAMPROW] ; inptr2
mov rdip, JSAMPROW [rdi] ; outptr
pop rcx ; col
.columnloop:
vmovdqu ymm6, YMMWORD [rbx]
; ymm6 = Cb(0123456789abcdefghijklmnopqrstuv)
vmovdqu ymm7, YMMWORD [rdx]
; ymm7 = Cr(0123456789abcdefghijklmnopqrstuv)
vpxor ymm1, ymm1, ymm1 ; ymm1 = (all 0's)
vpcmpeqw ymm3, ymm3, ymm3
vpsllw ymm3, ymm3, 7 ; ymm3 = { 0xFF80 0xFF80 0xFF80 0xFF80 .. }
vpermq ymm6, ymm6, 0xd8 ; ymm6 = Cb(01234567ghijklmn89abcdefopqrstuv)
vpermq ymm7, ymm7, 0xd8 ; ymm7 = Cr(01234567ghijklmn89abcdefopqrstuv)
vpunpcklbw ymm4, ymm6, ymm1 ; ymm4 = Cb(0123456789abcdef) = CbL
vpunpckhbw ymm6, ymm6, ymm1 ; ymm6 = Cb(ghijklmnopqrstuv) = CbH
vpunpcklbw ymm0, ymm7, ymm1 ; ymm0 = Cr(0123456789abcdef) = CrL
vpunpckhbw ymm7, ymm7, ymm1 ; ymm7 = Cr(ghijklmnopqrstuv) = CrH
vpaddw ymm5, ymm6, ymm3
vpaddw ymm2, ymm4, ymm3
vpaddw ymm1, ymm7, ymm3
vpaddw ymm3, ymm0, ymm3
; (Original)
; R = Y + 1.40200 * Cr
; G = Y - 0.34414 * Cb - 0.71414 * Cr
; B = Y + 1.77200 * Cb
;
; (This implementation)
; R = Y + 0.40200 * Cr + Cr
; G = Y - 0.34414 * Cb + 0.28586 * Cr - Cr
; B = Y - 0.22800 * Cb + Cb + Cb
vpaddw ymm6, ymm5, ymm5 ; ymm6 = 2 * CbH
vpaddw ymm4, ymm2, ymm2 ; ymm4 = 2 * CbL
vpaddw ymm7, ymm1, ymm1 ; ymm7 = 2 * CrH
vpaddw ymm0, ymm3, ymm3 ; ymm0 = 2 * CrL
vpmulhw ymm6, ymm6, [rel PW_MF0228] ; ymm6 = (2 * CbH * -FIX(0.22800))
vpmulhw ymm4, ymm4, [rel PW_MF0228] ; ymm4 = (2 * CbL * -FIX(0.22800))
vpmulhw ymm7, ymm7, [rel PW_F0402] ; ymm7 = (2 * CrH * FIX(0.40200))
vpmulhw ymm0, ymm0, [rel PW_F0402] ; ymm0 = (2 * CrL * FIX(0.40200))
vpaddw ymm6, ymm6, [rel PW_ONE]
vpaddw ymm4, ymm4, [rel PW_ONE]
vpsraw ymm6, ymm6, 1 ; ymm6 = (CbH * -FIX(0.22800))
vpsraw ymm4, ymm4, 1 ; ymm4 = (CbL * -FIX(0.22800))
vpaddw ymm7, ymm7, [rel PW_ONE]
vpaddw ymm0, ymm0, [rel PW_ONE]
vpsraw ymm7, ymm7, 1 ; ymm7 = (CrH * FIX(0.40200))
vpsraw ymm0, ymm0, 1 ; ymm0 = (CrL * FIX(0.40200))
vpaddw ymm6, ymm6, ymm5
vpaddw ymm4, ymm4, ymm2
vpaddw ymm6, ymm6, ymm5 ; ymm6 = (CbH * FIX(1.77200)) = (B - Y)H
vpaddw ymm4, ymm4, ymm2 ; ymm4 = (CbL * FIX(1.77200)) = (B - Y)L
vpaddw ymm7, ymm7, ymm1 ; ymm7 = (CrH * FIX(1.40200)) = (R - Y)H
vpaddw ymm0, ymm0, ymm3 ; ymm0 = (CrL * FIX(1.40200)) = (R - Y)L
vmovdqa YMMWORD [wk(0)], ymm6 ; wk(0) = (B - Y)H
vmovdqa YMMWORD [wk(1)], ymm7 ; wk(1) = (R - Y)H
vpunpckhwd ymm6, ymm5, ymm1
vpunpcklwd ymm5, ymm5, ymm1
vpmaddwd ymm5, ymm5, [rel PW_MF0344_F0285]
vpmaddwd ymm6, ymm6, [rel PW_MF0344_F0285]
vpunpckhwd ymm7, ymm2, ymm3
vpunpcklwd ymm2, ymm2, ymm3
vpmaddwd ymm2, ymm2, [rel PW_MF0344_F0285]
vpmaddwd ymm7, ymm7, [rel PW_MF0344_F0285]
vpaddd ymm5, ymm5, [rel PD_ONEHALF]
vpaddd ymm6, ymm6, [rel PD_ONEHALF]
vpsrad ymm5, ymm5, SCALEBITS
vpsrad ymm6, ymm6, SCALEBITS
vpaddd ymm2, ymm2, [rel PD_ONEHALF]
vpaddd ymm7, ymm7, [rel PD_ONEHALF]
vpsrad ymm2, ymm2, SCALEBITS
vpsrad ymm7, ymm7, SCALEBITS
vpackssdw ymm5, ymm5, ymm6
; ymm5 = CbH * -FIX(0.344) + CrH * FIX(0.285)
vpackssdw ymm2, ymm2, ymm7
; ymm2 = CbL * -FIX(0.344) + CrL * FIX(0.285)
vpsubw ymm5, ymm5, ymm1
; ymm5 = CbH * -FIX(0.344) + CrH * -FIX(0.714) = (G - Y)H
vpsubw ymm2, ymm2, ymm3
; ymm2 = CbL * -FIX(0.344) + CrL * -FIX(0.714) = (G - Y)L
vmovdqa YMMWORD [wk(2)], ymm5 ; wk(2) = (G - Y)H
mov al, 2 ; Yctr
jmp short .Yloop_1st
.Yloop_2nd:
vmovdqa ymm0, YMMWORD [wk(1)] ; ymm0 = (R - Y)H
vmovdqa ymm2, YMMWORD [wk(2)] ; ymm2 = (G - Y)H
vmovdqa ymm4, YMMWORD [wk(0)] ; ymm4 = (B - Y)H
.Yloop_1st:
vmovdqu ymm7, YMMWORD [rsi]
; ymm7 = Y(0123456789abcdefghijklmnopqrstuv)
vpcmpeqw ymm6, ymm6, ymm6
vpsrlw ymm6, ymm6, BYTE_BIT ; ymm6 = { 0xFF 0x00 0xFF 0x00 .. }
vpand ymm6, ymm6, ymm7 ; ymm6 = Y(02468acegikmoqsu) = YE
vpsrlw ymm7, ymm7, BYTE_BIT ; ymm7 = Y(13579bdfhjlnprtv) = YO
vmovdqa ymm1, ymm0 ; ymm1 = ymm0 = (R - Y)(L / H)
vmovdqa ymm3, ymm2 ; ymm3 = ymm2 = (G - Y)(L / H)
vmovdqa ymm5, ymm4 ; ymm5 = ymm4 = (B - Y)(L / H)
vpaddw ymm0, ymm0, ymm6
; ymm0 = ((R - Y) + YE) = RE = R(02468acegikmoqsu)
vpaddw ymm1, ymm1, ymm7
; ymm1 = ((R - Y) + YO) = RO = R(13579bdfhjlnprtv)
vpackuswb ymm0, ymm0, ymm0 ; ymm0 = R(02468ace********gikmoqsu********)
vpackuswb ymm1, ymm1, ymm1 ; ymm1 = R(13579bdf********hjlnprtv********)
vpaddw ymm2, ymm2, ymm6
; ymm2 = ((G - Y) + YE) = GE = G(02468acegikmoqsu)
vpaddw ymm3, ymm3, ymm7
; ymm3 = ((G - Y) + YO) = GO = G(13579bdfhjlnprtv)
vpackuswb ymm2, ymm2, ymm2 ; ymm2 = G(02468ace********gikmoqsu********)
vpackuswb ymm3, ymm3, ymm3 ; ymm3 = G(13579bdf********hjlnprtv********)
vpaddw ymm4, ymm4, ymm6
; ymm4 = ((B - Y) + YE) = BE = B(02468acegikmoqsu)
vpaddw ymm5, ymm5, ymm7
; ymm5 = ((B - Y) + YO) = BO = B(13579bdfhjlnprtv)
vpackuswb ymm4, ymm4, ymm4 ; ymm4 = B(02468ace********gikmoqsu********)
vpackuswb ymm5, ymm5, ymm5 ; ymm5 = B(13579bdf********hjlnprtv********)
%if RGB_PIXELSIZE == 3 ; ---------------
; NOTE: The values of RGB_RED, RGB_GREEN, and RGB_BLUE determine the
; mapping of components A, B, and C to red, green, and blue.
;
; ymmA = (A0 A2 A4 A6 A8 Aa Ac Ae Ag Ai Ak Am Ao Aq As Au) = AE
; ymmB = (A1 A3 A5 A7 A9 Ab Ad Af Ah Aj Al An Ap Ar At Av) = AO
; ymmC = (B0 B2 B4 B6 B8 Ba Bc Be Bg Bi Bk Bm Bo Bq Bs Bu) = BE
; ymmD = (B1 B3 B5 B7 B9 Bb Bd Bf Bh Bj Bl Bn Bp Br Bt Bv) = BO
; ymmE = (C0 C2 C4 C6 C8 Ca Cc Ce Cg Ci Ck Cm Co Cq Cs Cu) = CE
; ymmF = (C1 C3 C5 C7 C9 Cb Cd Cf Ch Cj Cl Cn Cp Cr Ct Cv) = CO
; ymmG = (** ** ** ** ** ** ** ** ** ** ** ** ** ** ** **)
; ymmH = (** ** ** ** ** ** ** ** ** ** ** ** ** ** ** **)
vpunpcklbw ymmA, ymmA, ymmC
; ymmA = (A0 B0 A2 B2 A4 B4 A6 B6 A8 B8 Aa Ba Ac Bc Ae Be
; Ag Bg Ai Bi Ak Bk Am Bm Ao Bo Aq Bq As Bs Au Bu)
vpunpcklbw ymmE, ymmE, ymmB
; ymmE = (C0 A1 C2 A3 C4 A5 C6 A7 C8 A9 Ca Ab Cc Ad Ce Af
; Cg Ah Ci Aj Ck Al Cm An Co Ap Cq Ar Cs At Cu Av)
vpunpcklbw ymmD, ymmD, ymmF
; ymmD = (B1 C1 B3 C3 B5 C5 B7 C7 B9 C9 Bb Cb Bd Cd Bf Cf
; Bh Ch Bj Cj Bl Cl Bn Cn Bp Cp Br Cr Bt Ct Bv Cv)
vpsrldq ymmH, ymmA, 2
; ymmH = (A2 B2 A4 B4 A6 B6 A8 B8 Aa Ba Ac Bc Ae Be Ag Bg
; Ai Bi Ak Bk Am Bm Ao Bo Aq Bq As Bs Au Bu -- --)
vpunpckhwd ymmG, ymmA, ymmE
; ymmG = (A8 B8 C8 A9 Aa Ba Ca Ab Ac Bc Cc Ad Ae Be Ce Af
; Ao Bo Co Ap Aq Bq Cq Ar As Bs Cs At Au Bu Cu Av)
vpunpcklwd ymmA, ymmA, ymmE
; ymmA = (A0 B0 C0 A1 A2 B2 C2 A3 A4 B4 C4 A5 A6 B6 C6 A7
; Ag Bg Cg Ah Ai Bi Ci Aj Ak Bk Ck Al Am Bm Cm An)
vpsrldq ymmE, ymmE, 2
; ymmE = (C2 A3 C4 A5 C6 A7 C8 A9 Ca Ab Cc Ad Ce Af Cg Ah
; Ci Aj Ck Al Cm An Co Ap Cq Ar Cs At Cu Av -- --)
vpsrldq ymmB, ymmD, 2
; ymmB = (B3 C3 B5 C5 B7 C7 B9 C9 Bb Cb Bd Cd Bf Cf Bh Ch
; Bj Cj Bl Cl Bn Cn Bp Cp Br Cr Bt Ct Bv Cv -- --)
vpunpckhwd ymmC, ymmD, ymmH
; ymmC = (B9 C9 Aa Ba Bb Cb Ac Bc Bd Cd Ae Be Bf Cf Ag Bg
; Bp Cp Aq Bq Br Cr As Bs Bt Ct Au Bu Bv Cv -- --)
vpunpcklwd ymmD, ymmD, ymmH
; ymmD = (B1 C1 A2 B2 B3 C3 A4 B4 B5 C5 A6 B6 B7 C7 A8 B8
; Bh Ch Ai Bi Bj Cj Ak Bk Bl Cl Am Bm Bn Cn Ao Bo)
vpunpckhwd ymmF, ymmE, ymmB
; ymmF = (Ca Ab Bb Cb Cc Ad Bd Cd Ce Af Bf Cf Cg Ah Bh Ch
; Cq Ar Br Cr Cs At Bt Ct Cu Av Bv Cv -- -- -- --)
vpunpcklwd ymmE, ymmE, ymmB
; ymmE = (C2 A3 B3 C3 C4 A5 B5 C5 C6 A7 B7 C7 C8 A9 B9 C9
; Ci Aj Bj Cj Ck Al Bl Cl Cm An Bn Cn Co Ap Bp Cp)
vpshufd ymmH, ymmA, 0x4E
; ymmH = (A4 B4 C4 A5 A6 B6 C6 A7 A0 B0 C0 A1 A2 B2 C2 A3
; Ak Bk Ck Al Am Bm Cm An Ag Bg Cg Ah Ai Bi Ci Aj)
vpunpckldq ymmA, ymmA, ymmD
; ymmA = (A0 B0 C0 A1 B1 C1 A2 B2 A2 B2 C2 A3 B3 C3 A4 B4
; Ag Bg Cg Ah Bh Ch Ai Bi Ai Bi Ci Aj Bj Cj Ak Bk)
vpunpckhdq ymmD, ymmD, ymmE
; ymmD = (B5 C5 A6 B6 C6 A7 B7 C7 B7 C7 A8 B8 C8 A9 B9 C9
; Bl Cl Am Bm Cm An Bn Cn Bn Cn Ao Bo Co Ap Bp Cp)
vpunpckldq ymmE, ymmE, ymmH
; ymmE = (C2 A3 B3 C3 A4 B4 C4 A5 C4 A5 B5 C5 A6 B6 C6 A7
; Ci Aj Bj Cj Ak Bk Ck Al Ck Al Bl Cl Am Bm Cm An)
vpshufd ymmH, ymmG, 0x4E
; ymmH = (Ac Bc Cc Ad Ae Be Ce Af A8 B8 C8 A9 Aa Ba Ca Ab
; As Bs Cs At Au Bu Cu Av Ao Bo Co Ap Aq Bq Cq Ar)
vpunpckldq ymmG, ymmG, ymmC
; ymmG = (A8 B8 C8 A9 B9 C9 Aa Ba Aa Ba Ca Ab Bb Cb Ac Bc
; Ao Bo Co Ap Bp Cp Aq Bq Aq Bq Cq Ar Br Cr As Bs)
vpunpckhdq ymmC, ymmC, ymmF
; ymmC = (Bd Cd Ae Be Ce Af Bf Cf Bf Cf Ag Bg Cg Ah Bh Ch
; Bt Ct Au Bu Cu Av Bv Cv Bv Cv -- -- -- -- -- --)
vpunpckldq ymmF, ymmF, ymmH
; ymmF = (Ca Ab Bb Cb Ac Bc Cc Ad Cc Ad Bd Cd Ae Be Ce Af
; Cq Ar Br Cr As Bs Cs At Cs At Bt Ct Au Bu Cu Av)
vpunpcklqdq ymmH, ymmA, ymmE
; ymmH = (A0 B0 C0 A1 B1 C1 A2 B2 C2 A3 B3 C3 A4 B4 C4 A5
; Ag Bg Cg Ah Bh Ch Ai Bi Ci Aj Bj Cj Ak Bk Ck Al)
vpunpcklqdq ymmG, ymmD, ymmG
; ymmG = (B5 C5 A6 B6 C6 A7 B7 C7 A8 B8 C8 A9 B9 C9 Aa Ba
; Bl Cl Am Bm Cm An Bn Cn Ao Bo Co Ap Bp Cp Aq Bq)
vpunpcklqdq ymmC, ymmF, ymmC
; ymmC = (Ca Ab Bb Cb Ac Bc Cc Ad Bd Cd Ae Be Ce Af Bf Cf
; Cq Ar Br Cr As Bs Cs At Bt Ct Au Bu Cu Av Bv Cv)
vperm2i128 ymmA, ymmH, ymmG, 0x20
; ymmA = (A0 B0 C0 A1 B1 C1 A2 B2 C2 A3 B3 C3 A4 B4 C4 A5
; B5 C5 A6 B6 C6 A7 B7 C7 A8 B8 C8 A9 B9 C9 Aa Ba)
vperm2i128 ymmD, ymmC, ymmH, 0x30
; ymmD = (Ca Ab Bb Cb Ac Bc Cc Ad Bd Cd Ae Be Ce Af Bf Cf
; Ag Bg Cg Ah Bh Ch Ai Bi Ci Aj Bj Cj Ak Bk Ck Al)
vperm2i128 ymmF, ymmG, ymmC, 0x31
; ymmF = (Bl Cl Am Bm Cm An Bn Cn Ao Bo Co Ap Bp Cp Aq Bq
; Cq Ar Br Cr As Bs Cs At Bt Ct Au Bu Cu Av Bv Cv)
cmp rcx, byte SIZEOF_YMMWORD
jb short .column_st64
test rdi, SIZEOF_YMMWORD - 1
jnz short .out1
; --(aligned)-------------------
vmovntdq YMMWORD [rdi + 0 * SIZEOF_YMMWORD], ymmA
vmovntdq YMMWORD [rdi + 1 * SIZEOF_YMMWORD], ymmD
vmovntdq YMMWORD [rdi + 2 * SIZEOF_YMMWORD], ymmF
jmp short .out0
.out1: ; --(unaligned)-----------------
vmovdqu YMMWORD [rdi + 0 * SIZEOF_YMMWORD], ymmA
vmovdqu YMMWORD [rdi + 1 * SIZEOF_YMMWORD], ymmD
vmovdqu YMMWORD [rdi + 2 * SIZEOF_YMMWORD], ymmF
.out0:
add rdi, byte RGB_PIXELSIZE * SIZEOF_YMMWORD ; outptr
sub rcx, byte SIZEOF_YMMWORD
jz near .endcolumn
add rsi, byte SIZEOF_YMMWORD ; inptr0
dec al ; Yctr
jnz near .Yloop_2nd
add rbx, byte SIZEOF_YMMWORD ; inptr1
add rdx, byte SIZEOF_YMMWORD ; inptr2
jmp near .columnloop
.column_st64:
lea rcx, [rcx + rcx * 2] ; imul ecx, RGB_PIXELSIZE
cmp rcx, byte 2 * SIZEOF_YMMWORD
jb short .column_st32
vmovdqu YMMWORD [rdi + 0 * SIZEOF_YMMWORD], ymmA
vmovdqu YMMWORD [rdi + 1 * SIZEOF_YMMWORD], ymmD
add rdi, byte 2 * SIZEOF_YMMWORD ; outptr
vmovdqa ymmA, ymmF
sub rcx, byte 2 * SIZEOF_YMMWORD
jmp short .column_st31
.column_st32:
cmp rcx, byte SIZEOF_YMMWORD
jb short .column_st31
vmovdqu YMMWORD [rdi + 0 * SIZEOF_YMMWORD], ymmA
add rdi, byte SIZEOF_YMMWORD ; outptr
vmovdqa ymmA, ymmD
sub rcx, byte SIZEOF_YMMWORD
jmp short .column_st31
.column_st31:
cmp rcx, byte SIZEOF_XMMWORD
jb short .column_st15
vmovdqu XMMWORD [rdi + 0 * SIZEOF_XMMWORD], xmmA
add rdi, byte SIZEOF_XMMWORD ; outptr
vperm2i128 ymmA, ymmA, ymmA, 1
sub rcx, byte SIZEOF_XMMWORD
.column_st15:
; Store the lower 8 bytes of xmmA to the output when it has enough
; space.
cmp rcx, byte SIZEOF_MMWORD
jb short .column_st7
vmovq XMM_MMWORD [rdi], xmmA
add rdi, byte SIZEOF_MMWORD
sub rcx, byte SIZEOF_MMWORD
vpsrldq xmmA, xmmA, SIZEOF_MMWORD
.column_st7:
; Store the lower 4 bytes of xmmA to the output when it has enough
; space.
cmp rcx, byte SIZEOF_DWORD
jb short .column_st3
vmovd XMM_DWORD [rdi], xmmA
add rdi, byte SIZEOF_DWORD
sub rcx, byte SIZEOF_DWORD
vpsrldq xmmA, xmmA, SIZEOF_DWORD
.column_st3:
; Store the lower 2 bytes of rax to the output when it has enough
; space.
vmovd eax, xmmA
cmp rcx, byte SIZEOF_WORD
jb short .column_st1
mov word [rdi], ax
add rdi, byte SIZEOF_WORD
sub rcx, byte SIZEOF_WORD
shr rax, 16
.column_st1:
; Store the lower 1 byte of rax to the output when it has enough
; space.
test rcx, rcx
jz short .endcolumn
mov byte [rdi], al
%else ; RGB_PIXELSIZE == 4 ; -----------
%ifdef RGBX_FILLER_0XFF
vpcmpeqb ymm6, ymm6, ymm6
; ymm6 = XE = X(02468ace********gikmoqsu********)
vpcmpeqb ymm7, ymm7, ymm7
; ymm7 = XO = X(13579bdf********hjlnprtv********)
%else
vpxor ymm6, ymm6, ymm6
; ymm6 = XE = X(02468ace********gikmoqsu********)
vpxor ymm7, ymm7, ymm7
; ymm7 = XO = X(13579bdf********hjlnprtv********)
%endif
; NOTE: The values of RGB_RED, RGB_GREEN, and RGB_BLUE determine the
; mapping of components A, B, C, and D to red, green, and blue.
;
; ymmA = (A0 A2 A4 A6 A8 Aa Ac Ae Ag Ai Ak Am Ao Aq As Au) = AE
; ymmB = (A1 A3 A5 A7 A9 Ab Ad Af Ah Aj Al An Ap Ar At Av) = AO
; ymmC = (B0 B2 B4 B6 B8 Ba Bc Be Bg Bi Bk Bm Bo Bq Bs Bu) = BE
; ymmD = (B1 B3 B5 B7 B9 Bb Bd Bf Bh Bj Bl Bn Bp Br Bt Bv) = BO
; ymmE = (C0 C2 C4 C6 C8 Ca Cc Ce Cg Ci Ck Cm Co Cq Cs Cu) = CE
; ymmF = (C1 C3 C5 C7 C9 Cb Cd Cf Ch Cj Cl Cn Cp Cr Ct Cv) = CO
; ymmG = (D0 D2 D4 D6 D8 Da Dc De Dg Di Dk Dm Do Dq Ds Du) = DE
; ymmH = (D1 D3 D5 D7 D9 Db Dd Df Dh Dj Dl Dn Dp Dr Dt Dv) = DO
vpunpcklbw ymmA, ymmA, ymmC
; ymmA = (A0 B0 A2 B2 A4 B4 A6 B6 A8 B8 Aa Ba Ac Bc Ae Be
; Ag Bg Ai Bi Ak Bk Am Bm Ao Bo Aq Bq As Bs Au Bu)
vpunpcklbw ymmE, ymmE, ymmG
; ymmE = (C0 D0 C2 D2 C4 D4 C6 D6 C8 D8 Ca Da Cc Dc Ce De
; Cg Dg Ci Di Ck Dk Cm Dm Co Do Cq Dq Cs Ds Cu Du)
vpunpcklbw ymmB, ymmB, ymmD
; ymmB = (A1 B1 A3 B3 A5 B5 A7 B7 A9 B9 Ab Bb Ad Bd Af Bf
; Ah Bh Aj Bj Al Bl An Bn Ap Bp Ar Br At Bt Av Bv)
vpunpcklbw ymmF, ymmF, ymmH
; ymmF = (C1 D1 C3 D3 C5 D5 C7 D7 C9 D9 Cb Db Cd Dd Cf Df
; Ch Dh Cj Dj Cl Dl Cn Dn Cp Dp Cr Dr Ct Dt Cv Dv)
vpunpckhwd ymmC, ymmA, ymmE
; ymmC = (A8 B8 C8 D8 Aa Ba Ca Da Ac Bc Cc Dc Ae Be Ce De
; Ao Bo Co Do Aq Bq Cq Dq As Bs Cs Ds Au Bu Cu Du)
vpunpcklwd ymmA, ymmA, ymmE
; ymmA = (A0 B0 C0 D0 A2 B2 C2 D2 A4 B4 C4 D4 A6 B6 C6 D6
; Ag Bg Cg Dg Ai Bi Ci Di Ak Bk Ck Dk Am Bm Cm Dm)
vpunpckhwd ymmG, ymmB, ymmF
; ymmG = (A9 B9 C9 D9 Ab Bb Cb Db Ad Bd Cd Dd Af Bf Cf Df
; Ap Bp Cp Dp Ar Br Cr Dr At Bt Ct Dt Av Bv Cv Dv)
vpunpcklwd ymmB, ymmB, ymmF
; ymmB = (A1 B1 C1 D1 A3 B3 C3 D3 A5 B5 C5 D5 A7 B7 C7 D7
; Ah Bh Ch Dh Aj Bj Cj Dj Al Bl Cl Dl An Bn Cn Dn)
vpunpckhdq ymmE, ymmA, ymmB
; ymmE = (A4 B4 C4 D4 A5 B5 C5 D5 A6 B6 C6 D6 A7 B7 C7 D7
; Ak Bk Ck Dk Al Bl Cl Dl Am Bm Cm Dm An Bn Cn Dn)
vpunpckldq ymmB, ymmA, ymmB
; ymmB = (A0 B0 C0 D0 A1 B1 C1 D1 A2 B2 C2 D2 A3 B3 C3 D3
; Ag Bg Cg Dg Ah Bh Ch Dh Ai Bi Ci Di Aj Bj Cj Dj)
vpunpckhdq ymmF, ymmC, ymmG
; ymmF = (Ac Bc Cc Dc Ad Bd Cd Dd Ae Be Ce De Af Bf Cf Df
; As Bs Cs Ds At Bt Ct Dt Au Bu Cu Du Av Bv Cv Dv)
vpunpckldq ymmG, ymmC, ymmG
; ymmG = (A8 B8 C8 D8 A9 B9 C9 D9 Aa Ba Ca Da Ab Bb Cb Db
; Ao Bo Co Do Ap Bp Cp Dp Aq Bq Cq Dq Ar Br Cr Dr)
vperm2i128 ymmA, ymmB, ymmE, 0x20
; ymmA = (A0 B0 C0 D0 A1 B1 C1 D1 A2 B2 C2 D2 A3 B3 C3 D3
; A4 B4 C4 D4 A5 B5 C5 D5 A6 B6 C6 D6 A7 B7 C7 D7)
vperm2i128 ymmD, ymmG, ymmF, 0x20
; ymmD = (A8 B8 C8 D8 A9 B9 C9 D9 Aa Ba Ca Da Ab Bb Cb Db
; Ac Bc Cc Dc Ad Bd Cd Dd Ae Be Ce De Af Bf Cf Df)
vperm2i128 ymmC, ymmB, ymmE, 0x31
; ymmC = (Ag Bg Cg Dg Ah Bh Ch Dh Ai Bi Ci Di Aj Bj Cj Dj
; Ak Bk Ck Dk Al Bl Cl Dl Am Bm Cm Dm An Bn Cn Dn)
vperm2i128 ymmH, ymmG, ymmF, 0x31
; ymmH = (Ao Bo Co Do Ap Bp Cp Dp Aq Bq Cq Dq Ar Br Cr Dr
; As Bs Cs Ds At Bt Ct Dt Au Bu Cu Du Av Bv Cv Dv)
cmp rcx, byte SIZEOF_YMMWORD
jb short .column_st64
test rdi, SIZEOF_YMMWORD - 1
jnz short .out1
; --(aligned)-------------------
vmovntdq YMMWORD [rdi + 0 * SIZEOF_YMMWORD], ymmA
vmovntdq YMMWORD [rdi + 1 * SIZEOF_YMMWORD], ymmD
vmovntdq YMMWORD [rdi + 2 * SIZEOF_YMMWORD], ymmC
vmovntdq YMMWORD [rdi + 3 * SIZEOF_YMMWORD], ymmH
jmp short .out0
.out1: ; --(unaligned)-----------------
vmovdqu YMMWORD [rdi + 0 * SIZEOF_YMMWORD], ymmA
vmovdqu YMMWORD [rdi + 1 * SIZEOF_YMMWORD], ymmD
vmovdqu YMMWORD [rdi + 2 * SIZEOF_YMMWORD], ymmC
vmovdqu YMMWORD [rdi + 3 * SIZEOF_YMMWORD], ymmH
.out0:
add rdi, RGB_PIXELSIZE * SIZEOF_YMMWORD ; outptr
sub rcx, byte SIZEOF_YMMWORD
jz near .endcolumn
add rsi, byte SIZEOF_YMMWORD ; inptr0
dec al
jnz near .Yloop_2nd
add rbx, byte SIZEOF_YMMWORD ; inptr1
add rdx, byte SIZEOF_YMMWORD ; inptr2
jmp near .columnloop
.column_st64:
cmp rcx, byte SIZEOF_YMMWORD / 2
jb short .column_st32
vmovdqu YMMWORD [rdi + 0 * SIZEOF_YMMWORD], ymmA
vmovdqu YMMWORD [rdi + 1 * SIZEOF_YMMWORD], ymmD
add rdi, byte 2 * SIZEOF_YMMWORD ; outptr
vmovdqa ymmA, ymmC
vmovdqa ymmD, ymmH
sub rcx, byte SIZEOF_YMMWORD / 2
.column_st32:
cmp rcx, byte SIZEOF_YMMWORD / 4
jb short .column_st16
vmovdqu YMMWORD [rdi + 0 * SIZEOF_YMMWORD], ymmA
add rdi, byte SIZEOF_YMMWORD ; outptr
vmovdqa ymmA, ymmD
sub rcx, byte SIZEOF_YMMWORD / 4
.column_st16:
cmp rcx, byte SIZEOF_YMMWORD / 8
jb short .column_st15
vmovdqu XMMWORD [rdi + 0 * SIZEOF_XMMWORD], xmmA
add rdi, byte SIZEOF_XMMWORD ; outptr
vperm2i128 ymmA, ymmA, ymmA, 1
sub rcx, byte SIZEOF_YMMWORD / 8
.column_st15:
; Store two pixels (8 bytes) of ymmA to the output when it has enough
; space.
cmp rcx, byte SIZEOF_YMMWORD / 16
jb short .column_st7
vmovq MMWORD [rdi], xmmA
add rdi, byte SIZEOF_YMMWORD / 16 * 4
sub rcx, byte SIZEOF_YMMWORD / 16
vpsrldq xmmA, SIZEOF_YMMWORD / 16 * 4
.column_st7:
; Store one pixel (4 bytes) of ymmA to the output when it has enough
; space.
test rcx, rcx
jz short .endcolumn
vmovd XMM_DWORD [rdi], xmmA
%endif ; RGB_PIXELSIZE ; ---------------
.endcolumn:
sfence ; flush the write buffer
.return:
pop rbx
vzeroupper
UNCOLLECT_ARGS 4
lea rsp, [rbp - 8]
pop r15
pop rbp
ret
; --------------------------------------------------------------------------
;
; Upsample and color convert for the case of 2:1 horizontal and 2:1 vertical.
;
; GLOBAL(void)
; jsimd_h2v2_merged_upsample_avx2(JDIMENSION output_width,
; JSAMPIMAGE input_buf,
; JDIMENSION in_row_group_ctr,
; JSAMPARRAY output_buf)
;
; r10d = JDIMENSION output_width
; r11 = JSAMPIMAGE input_buf
; r12d = JDIMENSION in_row_group_ctr
; r13 = JSAMPARRAY output_buf
align 32
GLOBAL_FUNCTION(jsimd_h2v2_merged_upsample_avx2)
EXTN(jsimd_h2v2_merged_upsample_avx2):
ENDBR64
push rbp
mov rbp, rsp
COLLECT_ARGS 4
push rbx
mov eax, r10d
mov rdi, r11
mov ecx, r12d
mov rsip, JSAMPARRAY [rdi + 0 * SIZEOF_JSAMPARRAY]
mov rbxp, JSAMPARRAY [rdi + 1 * SIZEOF_JSAMPARRAY]
mov rdxp, JSAMPARRAY [rdi + 2 * SIZEOF_JSAMPARRAY]
mov rdi, r13
lea rsi, [rsi + rcx * SIZEOF_JSAMPROW]
sub rsp, SIZEOF_JSAMPARRAY * 4
mov JSAMPARRAY [rsp + 0 * SIZEOF_JSAMPARRAY], rsip ; intpr00
mov JSAMPARRAY [rsp + 1 * SIZEOF_JSAMPARRAY], rbxp ; intpr1
mov JSAMPARRAY [rsp + 2 * SIZEOF_JSAMPARRAY], rdxp ; intpr2
mov rbx, rsp
push rdi
push rcx
push rax
%ifdef WIN64
mov r8, rcx
mov r9, rdi
mov rcx, rax
mov rdx, rbx
%else
mov rdx, rcx
mov rcx, rdi
mov rdi, rax
mov rsi, rbx
%endif
call EXTN(jsimd_h2v1_merged_upsample_avx2)
pop rax
pop rcx
pop rdi
mov rsip, JSAMPARRAY [rsp + 0 * SIZEOF_JSAMPARRAY]
mov rbxp, JSAMPARRAY [rsp + 1 * SIZEOF_JSAMPARRAY]
mov rdxp, JSAMPARRAY [rsp + 2 * SIZEOF_JSAMPARRAY]
add rdi, byte SIZEOF_JSAMPROW ; outptr1
add rsi, byte SIZEOF_JSAMPROW ; inptr01
mov JSAMPARRAY [rsp + 0 * SIZEOF_JSAMPARRAY], rsip ; intpr00
mov JSAMPARRAY [rsp + 1 * SIZEOF_JSAMPARRAY], rbxp ; intpr1
mov JSAMPARRAY [rsp + 2 * SIZEOF_JSAMPARRAY], rdxp ; intpr2
mov rbx, rsp
push rdi
push rcx
push rax
%ifdef WIN64
mov r8, rcx
mov r9, rdi
mov rcx, rax
mov rdx, rbx
%else
mov rdx, rcx
mov rcx, rdi
mov rdi, rax
mov rsi, rbx
%endif
call EXTN(jsimd_h2v1_merged_upsample_avx2)
pop rax
pop rcx
pop rdi
mov rsip, JSAMPARRAY [rsp + 0 * SIZEOF_JSAMPARRAY]
mov rbxp, JSAMPARRAY [rsp + 1 * SIZEOF_JSAMPARRAY]
mov rdxp, JSAMPARRAY [rsp + 2 * SIZEOF_JSAMPARRAY]
add rsp, SIZEOF_JSAMPARRAY * 4
pop rbx
UNCOLLECT_ARGS 4
pop rbp
ret
; For some reason, the OS X linker does not honor the request to align the
; segment unless we do this.
align 32
lib/libjpeg-turbo/simd/x86_64/jdmrgext-sse2.asm
Executable
+587
View File
@@ -0,0 +1,587 @@
;
; Merged upsampling/color conversion (64-bit SSE2)
;
; Copyright 2009, 2012 Pierre Ossman <[email protected]> for Cendio AB
; Copyright (C) 2009, 2012, 2016, 2024-2025, D. R. Commander.
; Copyright (C) 2018, Matthias Räncker.
; Copyright (C) 2023, Aliaksiej Kandracienka.
;
; Based on the x86 SIMD extension for IJG JPEG library
; Copyright (C) 1999-2006, MIYASAKA Masaru.
; For conditions of distribution and use, see copyright notice in jsimdext.inc
;
; This file should be assembled with NASM (Netwide Assembler) or Yasm.
%include "jcolsamp.inc"
; --------------------------------------------------------------------------
;
; Upsample and color convert for the case of 2:1 horizontal and 1:1 vertical.
;
; GLOBAL(void)
; jsimd_h2v1_merged_upsample_sse2(JDIMENSION output_width,
; JSAMPIMAGE input_buf,
; JDIMENSION in_row_group_ctr,
; JSAMPARRAY output_buf)
;
; r10d = JDIMENSION output_width
; r11 = JSAMPIMAGE input_buf
; r12d = JDIMENSION in_row_group_ctr
; r13 = JSAMPARRAY output_buf
%define wk(i) r15 - (WK_NUM - (i)) * SIZEOF_XMMWORD ; xmmword wk[WK_NUM]
%define WK_NUM 3
align 32
GLOBAL_FUNCTION(jsimd_h2v1_merged_upsample_sse2)
EXTN(jsimd_h2v1_merged_upsample_sse2):
ENDBR64
push rbp
mov rbp, rsp
push r15
and rsp, byte (-SIZEOF_XMMWORD) ; align to 128 bits
; Allocate stack space for wk array. r15 is used to access it.
mov r15, rsp
sub rsp, byte (SIZEOF_XMMWORD * WK_NUM)
COLLECT_ARGS 4
push rbx
mov ecx, r10d ; col
test rcx, rcx
jz near .return
push rcx
mov rdi, r11
mov ecx, r12d
mov rsip, JSAMPARRAY [rdi + 0 * SIZEOF_JSAMPARRAY]
mov rbxp, JSAMPARRAY [rdi + 1 * SIZEOF_JSAMPARRAY]
mov rdxp, JSAMPARRAY [rdi + 2 * SIZEOF_JSAMPARRAY]
mov rdi, r13
mov rsip, JSAMPROW [rsi + rcx * SIZEOF_JSAMPROW] ; inptr0
mov rbxp, JSAMPROW [rbx + rcx * SIZEOF_JSAMPROW] ; inptr1
mov rdxp, JSAMPROW [rdx + rcx * SIZEOF_JSAMPROW] ; inptr2
mov rdip, JSAMPROW [rdi] ; outptr
pop rcx ; col
.columnloop:
movdqa xmm6, XMMWORD [rbx] ; xmm6 = Cb(0123456789abcdef)
movdqa xmm7, XMMWORD [rdx] ; xmm7 = Cr(0123456789abcdef)
pxor xmm1, xmm1 ; xmm1 = (all 0's)
pcmpeqw xmm3, xmm3
psllw xmm3, 7 ; xmm3 = { 0xFF80 0xFF80 0xFF80 0xFF80 .. }
movdqa xmm4, xmm6
punpckhbw xmm6, xmm1 ; xmm6 = Cb(89abcdef) = CbH
punpcklbw xmm4, xmm1 ; xmm4 = Cb(01234567) = CbL
movdqa xmm0, xmm7
punpckhbw xmm7, xmm1 ; xmm7 = Cr(89abcdef) = CrH
punpcklbw xmm0, xmm1 ; xmm0 = Cr(01234567) = CrL
paddw xmm6, xmm3
paddw xmm4, xmm3
paddw xmm7, xmm3
paddw xmm0, xmm3
; (Original)
; R = Y + 1.40200 * Cr
; G = Y - 0.34414 * Cb - 0.71414 * Cr
; B = Y + 1.77200 * Cb
;
; (This implementation)
; R = Y + 0.40200 * Cr + Cr
; G = Y - 0.34414 * Cb + 0.28586 * Cr - Cr
; B = Y - 0.22800 * Cb + Cb + Cb
movdqa xmm5, xmm6 ; xmm5 = CbH
movdqa xmm2, xmm4 ; xmm2 = CbL
paddw xmm6, xmm6 ; xmm6 = 2 * CbH
paddw xmm4, xmm4 ; xmm4 = 2 * CbL
movdqa xmm1, xmm7 ; xmm1 = CrH
movdqa xmm3, xmm0 ; xmm3 = CrL
paddw xmm7, xmm7 ; xmm7 = 2 * CrH
paddw xmm0, xmm0 ; xmm0 = 2 * CrL
pmulhw xmm6, [rel PW_MF0228] ; xmm6 = (2 * CbH * -FIX(0.22800))
pmulhw xmm4, [rel PW_MF0228] ; xmm4 = (2 * CbL * -FIX(0.22800))
pmulhw xmm7, [rel PW_F0402] ; xmm7 = (2 * CrH * FIX(0.40200))
pmulhw xmm0, [rel PW_F0402] ; xmm0 = (2 * CrL * FIX(0.40200))
paddw xmm6, [rel PW_ONE]
paddw xmm4, [rel PW_ONE]
psraw xmm6, 1 ; xmm6 = (CbH * -FIX(0.22800))
psraw xmm4, 1 ; xmm4 = (CbL * -FIX(0.22800))
paddw xmm7, [rel PW_ONE]
paddw xmm0, [rel PW_ONE]
psraw xmm7, 1 ; xmm7 = (CrH * FIX(0.40200))
psraw xmm0, 1 ; xmm0 = (CrL * FIX(0.40200))
paddw xmm6, xmm5
paddw xmm4, xmm2
paddw xmm6, xmm5 ; xmm6 = (CbH * FIX(1.77200)) = (B - Y)H
paddw xmm4, xmm2 ; xmm4 = (CbL * FIX(1.77200)) = (B - Y)L
paddw xmm7, xmm1 ; xmm7 = (CrH * FIX(1.40200)) = (R - Y)H
paddw xmm0, xmm3 ; xmm0 = (CrL * FIX(1.40200)) = (R - Y)L
movdqa XMMWORD [wk(0)], xmm6 ; wk(0) = (B - Y)H
movdqa XMMWORD [wk(1)], xmm7 ; wk(1) = (R - Y)H
movdqa xmm6, xmm5
movdqa xmm7, xmm2
punpcklwd xmm5, xmm1
punpckhwd xmm6, xmm1
pmaddwd xmm5, [rel PW_MF0344_F0285]
pmaddwd xmm6, [rel PW_MF0344_F0285]
punpcklwd xmm2, xmm3
punpckhwd xmm7, xmm3
pmaddwd xmm2, [rel PW_MF0344_F0285]
pmaddwd xmm7, [rel PW_MF0344_F0285]
paddd xmm5, [rel PD_ONEHALF]
paddd xmm6, [rel PD_ONEHALF]
psrad xmm5, SCALEBITS
psrad xmm6, SCALEBITS
paddd xmm2, [rel PD_ONEHALF]
paddd xmm7, [rel PD_ONEHALF]
psrad xmm2, SCALEBITS
psrad xmm7, SCALEBITS
packssdw xmm5, xmm6
; xmm5 = CbH * -FIX(0.344) + CrH * FIX(0.285)
packssdw xmm2, xmm7
; xmm2 = CbL * -FIX(0.344) + CrL * FIX(0.285)
psubw xmm5, xmm1
; xmm5 = CbH * -FIX(0.344) + CrH * -FIX(0.714) = (G - Y)H
psubw xmm2, xmm3
; xmm2 = CbL * -FIX(0.344) + CrL * -FIX(0.714) = (G - Y)L
movdqa XMMWORD [wk(2)], xmm5 ; wk(2) = (G - Y)H
mov al, 2 ; Yctr
jmp short .Yloop_1st
.Yloop_2nd:
movdqa xmm0, XMMWORD [wk(1)] ; xmm0 = (R - Y)H
movdqa xmm2, XMMWORD [wk(2)] ; xmm2 = (G - Y)H
movdqa xmm4, XMMWORD [wk(0)] ; xmm4 = (B - Y)H
.Yloop_1st:
movdqa xmm7, XMMWORD [rsi] ; xmm7 = Y(0123456789abcdef)
pcmpeqw xmm6, xmm6
psrlw xmm6, BYTE_BIT ; xmm6 = { 0xFF 0x00 0xFF 0x00 .. }
pand xmm6, xmm7 ; xmm6 = Y(02468ace) = YE
psrlw xmm7, BYTE_BIT ; xmm7 = Y(13579bdf) = YO
movdqa xmm1, xmm0 ; xmm1 = xmm0 = (R - Y)(L / H)
movdqa xmm3, xmm2 ; xmm3 = xmm2 = (G - Y)(L / H)
movdqa xmm5, xmm4 ; xmm5 = xmm4 = (B - Y)(L / H)
paddw xmm0, xmm6 ; xmm0 = ((R - Y) + YE) = RE = R(02468ace)
paddw xmm1, xmm7 ; xmm1 = ((R - Y) + YO) = RO = R(13579bdf)
packuswb xmm0, xmm0 ; xmm0 = R(02468ace********)
packuswb xmm1, xmm1 ; xmm1 = R(13579bdf********)
paddw xmm2, xmm6 ; xmm2 = ((G - Y) + YE) = GE = G(02468ace)
paddw xmm3, xmm7 ; xmm3 = ((G - Y) + YO) = GO = G(13579bdf)
packuswb xmm2, xmm2 ; xmm2 = G(02468ace********)
packuswb xmm3, xmm3 ; xmm3 = G(13579bdf********)
paddw xmm4, xmm6 ; xmm4 = ((B - Y) + YE) = BE = B(02468ace)
paddw xmm5, xmm7 ; xmm5 = ((B - Y) + YO) = BO = B(13579bdf)
packuswb xmm4, xmm4 ; xmm4 = B(02468ace********)
packuswb xmm5, xmm5 ; xmm5 = B(13579bdf********)
%if RGB_PIXELSIZE == 3 ; ---------------
; NOTE: The values of RGB_RED, RGB_GREEN, and RGB_BLUE determine the
; mapping of components A, B, and C to red, green, and blue.
;
; xmmA = (A0 A2 A4 A6 A8 Aa Ac Ae) = AE
; xmmB = (A1 A3 A5 A7 A9 Ab Ad Af) = AO
; xmmC = (B0 B2 B4 B6 B8 Ba Bc Be) = BE
; xmmD = (B1 B3 B5 B7 B9 Bb Bd Bf) = BO
; xmmE = (C0 C2 C4 C6 C8 Ca Cc Ce) = CE
; xmmF = (C1 C3 C5 C7 C9 Cb Cd Cf) = CO
; xmmG = (** ** ** ** ** ** ** **)
; xmmH = (** ** ** ** ** ** ** **)
punpcklbw xmmA, xmmC
; xmmA = (A0 B0 A2 B2 A4 B4 A6 B6 A8 B8 Aa Ba Ac Bc Ae Be)
punpcklbw xmmE, xmmB
; xmmE = (C0 A1 C2 A3 C4 A5 C6 A7 C8 A9 Ca Ab Cc Ad Ce Af)
punpcklbw xmmD, xmmF
; xmmD = (B1 C1 B3 C3 B5 C5 B7 C7 B9 C9 Bb Cb Bd Cd Bf Cf)
movdqa xmmG, xmmA
movdqa xmmH, xmmA
punpcklwd xmmA, xmmE
; xmmA = (A0 B0 C0 A1 A2 B2 C2 A3 A4 B4 C4 A5 A6 B6 C6 A7)
punpckhwd xmmG, xmmE
; xmmG = (A8 B8 C8 A9 Aa Ba Ca Ab Ac Bc Cc Ad Ae Be Ce Af)
psrldq xmmH, 2
; xmmH = (A2 B2 A4 B4 A6 B6 A8 B8 Aa Ba Ac Bc Ae Be -- --)
psrldq xmmE, 2
; xmmE = (C2 A3 C4 A5 C6 A7 C8 A9 Ca Ab Cc Ad Ce Af -- --)
movdqa xmmC, xmmD
movdqa xmmB, xmmD
punpcklwd xmmD, xmmH
; xmmD = (B1 C1 A2 B2 B3 C3 A4 B4 B5 C5 A6 B6 B7 C7 A8 B8)
punpckhwd xmmC, xmmH
; xmmC = (B9 C9 Aa Ba Bb Cb Ac Bc Bd Cd Ae Be Bf Cf -- --)
psrldq xmmB, 2
; xmmB = (B3 C3 B5 C5 B7 C7 B9 C9 Bb Cb Bd Cd Bf Cf -- --)
movdqa xmmF, xmmE
punpcklwd xmmE, xmmB
; xmmE = (C2 A3 B3 C3 C4 A5 B5 C5 C6 A7 B7 C7 C8 A9 B9 C9)
punpckhwd xmmF, xmmB
; xmmF = (Ca Ab Bb Cb Cc Ad Bd Cd Ce Af Bf Cf -- -- -- --)
pshufd xmmH, xmmA, 0x4E
; xmmH = (A4 B4 C4 A5 A6 B6 C6 A7 A0 B0 C0 A1 A2 B2 C2 A3)
movdqa xmmB, xmmE
punpckldq xmmA, xmmD
; xmmA = (A0 B0 C0 A1 B1 C1 A2 B2 A2 B2 C2 A3 B3 C3 A4 B4)
punpckldq xmmE, xmmH
; xmmE = (C2 A3 B3 C3 A4 B4 C4 A5 C4 A5 B5 C5 A6 B6 C6 A7)
punpckhdq xmmD, xmmB
; xmmD = (B5 C5 A6 B6 C6 A7 B7 C7 B7 C7 A8 B8 C8 A9 B9 C9)
pshufd xmmH, xmmG, 0x4E
; xmmH = (Ac Bc Cc Ad Ae Be Ce Af A8 B8 C8 A9 Aa Ba Ca Ab)
movdqa xmmB, xmmF
punpckldq xmmG, xmmC
; xmmG = (A8 B8 C8 A9 B9 C9 Aa Ba Aa Ba Ca Ab Bb Cb Ac Bc)
punpckldq xmmF, xmmH
; xmmF = (Ca Ab Bb Cb Ac Bc Cc Ad Cc Ad Bd Cd Ae Be Ce Af)
punpckhdq xmmC, xmmB
; xmmC = (Bd Cd Ae Be Ce Af Bf Cf Bf Cf -- -- -- -- -- --)
punpcklqdq xmmA, xmmE
; xmmA = (A0 B0 C0 A1 B1 C1 A2 B2 C2 A3 B3 C3 A4 B4 C4 A5)
punpcklqdq xmmD, xmmG
; xmmD = (B5 C5 A6 B6 C6 A7 B7 C7 A8 B8 C8 A9 B9 C9 Aa Ba)
punpcklqdq xmmF, xmmC
; xmmF = (Ca Ab Bb Cb Ac Bc Cc Ad Bd Cd Ae Be Ce Af Bf Cf)
cmp rcx, byte SIZEOF_XMMWORD
jb short .column_st32
test rdi, SIZEOF_XMMWORD - 1
jnz short .out1
; --(aligned)-------------------
movntdq XMMWORD [rdi + 0 * SIZEOF_XMMWORD], xmmA
movntdq XMMWORD [rdi + 1 * SIZEOF_XMMWORD], xmmD
movntdq XMMWORD [rdi + 2 * SIZEOF_XMMWORD], xmmF
jmp short .out0
.out1: ; --(unaligned)-----------------
movdqu XMMWORD [rdi + 0 * SIZEOF_XMMWORD], xmmA
movdqu XMMWORD [rdi + 1 * SIZEOF_XMMWORD], xmmD
movdqu XMMWORD [rdi + 2 * SIZEOF_XMMWORD], xmmF
.out0:
add rdi, byte RGB_PIXELSIZE * SIZEOF_XMMWORD ; outptr
sub rcx, byte SIZEOF_XMMWORD
jz near .endcolumn
add rsi, byte SIZEOF_XMMWORD ; inptr0
dec al ; Yctr
jnz near .Yloop_2nd
add rbx, byte SIZEOF_XMMWORD ; inptr1
add rdx, byte SIZEOF_XMMWORD ; inptr2
jmp near .columnloop
.column_st32:
lea rcx, [rcx + rcx * 2] ; imul ecx, RGB_PIXELSIZE
cmp rcx, byte 2 * SIZEOF_XMMWORD
jb short .column_st16
movdqu XMMWORD [rdi + 0 * SIZEOF_XMMWORD], xmmA
movdqu XMMWORD [rdi + 1 * SIZEOF_XMMWORD], xmmD
add rdi, byte 2 * SIZEOF_XMMWORD ; outptr
movdqa xmmA, xmmF
sub rcx, byte 2 * SIZEOF_XMMWORD
jmp short .column_st15
.column_st16:
cmp rcx, byte SIZEOF_XMMWORD
jb short .column_st15
movdqu XMMWORD [rdi + 0 * SIZEOF_XMMWORD], xmmA
add rdi, byte SIZEOF_XMMWORD ; outptr
movdqa xmmA, xmmD
sub rcx, byte SIZEOF_XMMWORD
.column_st15:
; Store the lower 8 bytes of xmmA to the output when it has enough
; space.
cmp rcx, byte SIZEOF_MMWORD
jb short .column_st7
movq XMM_MMWORD [rdi], xmmA
add rdi, byte SIZEOF_MMWORD
sub rcx, byte SIZEOF_MMWORD
psrldq xmmA, SIZEOF_MMWORD
.column_st7:
; Store the lower 4 bytes of xmmA to the output when it has enough
; space.
cmp rcx, byte SIZEOF_DWORD
jb short .column_st3
movd XMM_DWORD [rdi], xmmA
add rdi, byte SIZEOF_DWORD
sub rcx, byte SIZEOF_DWORD
psrldq xmmA, SIZEOF_DWORD
.column_st3:
; Store the lower 2 bytes of rax to the output when it has enough
; space.
movd eax, xmmA
cmp rcx, byte SIZEOF_WORD
jb short .column_st1
mov word [rdi], ax
add rdi, byte SIZEOF_WORD
sub rcx, byte SIZEOF_WORD
shr rax, 16
.column_st1:
; Store the lower 1 byte of rax to the output when it has enough
; space.
test rcx, rcx
jz short .endcolumn
mov byte [rdi], al
%else ; RGB_PIXELSIZE == 4 ; -----------
%ifdef RGBX_FILLER_0XFF
pcmpeqb xmm6, xmm6 ; xmm6 = XE = X(02468ace********)
pcmpeqb xmm7, xmm7 ; xmm7 = XO = X(13579bdf********)
%else
pxor xmm6, xmm6 ; xmm6 = XE = X(02468ace********)
pxor xmm7, xmm7 ; xmm7 = XO = X(13579bdf********)
%endif
; NOTE: The values of RGB_RED, RGB_GREEN, and RGB_BLUE determine the
; mapping of components A, B, C, and D to red, green, and blue.
;
; xmmA = (A0 A2 A4 A6 A8 Aa Ac Ae) = AE
; xmmB = (A1 A3 A5 A7 A9 Ab Ad Af) = AO
; xmmC = (B0 B2 B4 B6 B8 Ba Bc Be) = BE
; xmmD = (B1 B3 B5 B7 B9 Bb Bd Bf) = BO
; xmmE = (C0 C2 C4 C6 C8 Ca Cc Ce) = CE
; xmmF = (C1 C3 C5 C7 C9 Cb Cd Cf) = CO
; xmmG = (D0 D2 D4 D6 D8 Da Dc De) = DE
; xmmH = (D1 D3 D5 D7 D9 Db Dd Df) = DO
punpcklbw xmmA, xmmC
; xmmA = (A0 B0 A2 B2 A4 B4 A6 B6 A8 B8 Aa Ba Ac Bc Ae Be)
punpcklbw xmmE, xmmG
; xmmE = (C0 D0 C2 D2 C4 D4 C6 D6 C8 D8 Ca Da Cc Dc Ce De)
punpcklbw xmmB, xmmD
; xmmB = (A1 B1 A3 B3 A5 B5 A7 B7 A9 B9 Ab Bb Ad Bd Af Bf)
punpcklbw xmmF, xmmH
; xmmF = (C1 D1 C3 D3 C5 D5 C7 D7 C9 D9 Cb Db Cd Dd Cf Df)
movdqa xmmC, xmmA
punpcklwd xmmA, xmmE
; xmmA = (A0 B0 C0 D0 A2 B2 C2 D2 A4 B4 C4 D4 A6 B6 C6 D6)
punpckhwd xmmC, xmmE
; xmmC = (A8 B8 C8 D8 Aa Ba Ca Da Ac Bc Cc Dc Ae Be Ce De)
movdqa xmmG, xmmB
punpcklwd xmmB, xmmF
; xmmB = (A1 B1 C1 D1 A3 B3 C3 D3 A5 B5 C5 D5 A7 B7 C7 D7)
punpckhwd xmmG, xmmF
; xmmG = (A9 B9 C9 D9 Ab Bb Cb Db Ad Bd Cd Dd Af Bf Cf Df)
movdqa xmmD, xmmA
punpckldq xmmA, xmmB
; xmmA = (A0 B0 C0 D0 A1 B1 C1 D1 A2 B2 C2 D2 A3 B3 C3 D3)
punpckhdq xmmD, xmmB
; xmmD = (A4 B4 C4 D4 A5 B5 C5 D5 A6 B6 C6 D6 A7 B7 C7 D7)
movdqa xmmH, xmmC
punpckldq xmmC, xmmG
; xmmC = (A8 B8 C8 D8 A9 B9 C9 D9 Aa Ba Ca Da Ab Bb Cb Db)
punpckhdq xmmH, xmmG
; xmmH = (Ac Bc Cc Dc Ad Bd Cd Dd Ae Be Ce De Af Bf Cf Df)
cmp rcx, byte SIZEOF_XMMWORD
jb short .column_st32
test rdi, SIZEOF_XMMWORD - 1
jnz short .out1
; --(aligned)-------------------
movntdq XMMWORD [rdi + 0 * SIZEOF_XMMWORD], xmmA
movntdq XMMWORD [rdi + 1 * SIZEOF_XMMWORD], xmmD
movntdq XMMWORD [rdi + 2 * SIZEOF_XMMWORD], xmmC
movntdq XMMWORD [rdi + 3 * SIZEOF_XMMWORD], xmmH
jmp short .out0
.out1: ; --(unaligned)-----------------
movdqu XMMWORD [rdi + 0 * SIZEOF_XMMWORD], xmmA
movdqu XMMWORD [rdi + 1 * SIZEOF_XMMWORD], xmmD
movdqu XMMWORD [rdi + 2 * SIZEOF_XMMWORD], xmmC
movdqu XMMWORD [rdi + 3 * SIZEOF_XMMWORD], xmmH
.out0:
add rdi, byte RGB_PIXELSIZE * SIZEOF_XMMWORD ; outptr
sub rcx, byte SIZEOF_XMMWORD
jz near .endcolumn
add rsi, byte SIZEOF_XMMWORD ; inptr0
dec al ; Yctr
jnz near .Yloop_2nd
add rbx, byte SIZEOF_XMMWORD ; inptr1
add rdx, byte SIZEOF_XMMWORD ; inptr2
jmp near .columnloop
.column_st32:
cmp rcx, byte SIZEOF_XMMWORD / 2
jb short .column_st16
movdqu XMMWORD [rdi + 0 * SIZEOF_XMMWORD], xmmA
movdqu XMMWORD [rdi + 1 * SIZEOF_XMMWORD], xmmD
add rdi, byte 2 * SIZEOF_XMMWORD ; outptr
movdqa xmmA, xmmC
movdqa xmmD, xmmH
sub rcx, byte SIZEOF_XMMWORD / 2
.column_st16:
cmp rcx, byte SIZEOF_XMMWORD / 4
jb short .column_st15
movdqu XMMWORD [rdi + 0 * SIZEOF_XMMWORD], xmmA
add rdi, byte SIZEOF_XMMWORD ; outptr
movdqa xmmA, xmmD
sub rcx, byte SIZEOF_XMMWORD / 4
.column_st15:
; Store two pixels (8 bytes) of xmmA to the output when it has enough
; space.
cmp rcx, byte SIZEOF_XMMWORD / 8
jb short .column_st7
movq XMM_MMWORD [rdi], xmmA
add rdi, byte SIZEOF_XMMWORD / 8 * 4
sub rcx, byte SIZEOF_XMMWORD / 8
psrldq xmmA, SIZEOF_XMMWORD / 8 * 4
.column_st7:
; Store one pixel (4 bytes) of xmmA to the output when it has enough
; space.
test rcx, rcx
jz short .endcolumn
movd XMM_DWORD [rdi], xmmA
%endif ; RGB_PIXELSIZE ; ---------------
.endcolumn:
sfence ; flush the write buffer
.return:
pop rbx
UNCOLLECT_ARGS 4
lea rsp, [rbp - 8]
pop r15
pop rbp
ret
; --------------------------------------------------------------------------
;
; Upsample and color convert for the case of 2:1 horizontal and 2:1 vertical.
;
; GLOBAL(void)
; jsimd_h2v2_merged_upsample_sse2(JDIMENSION output_width,
; JSAMPIMAGE input_buf,
; JDIMENSION in_row_group_ctr,
; JSAMPARRAY output_buf)
;
; r10d = JDIMENSION output_width
; r11 = JSAMPIMAGE input_buf
; r12d = JDIMENSION in_row_group_ctr
; r13 = JSAMPARRAY output_buf
align 32
GLOBAL_FUNCTION(jsimd_h2v2_merged_upsample_sse2)
EXTN(jsimd_h2v2_merged_upsample_sse2):
ENDBR64
push rbp
mov rbp, rsp
COLLECT_ARGS 4
push rbx
mov eax, r10d
mov rdi, r11
mov ecx, r12d
mov rsip, JSAMPARRAY [rdi + 0 * SIZEOF_JSAMPARRAY]
mov rbxp, JSAMPARRAY [rdi + 1 * SIZEOF_JSAMPARRAY]
mov rdxp, JSAMPARRAY [rdi + 2 * SIZEOF_JSAMPARRAY]
mov rdi, r13
lea rsi, [rsi + rcx * SIZEOF_JSAMPROW]
sub rsp, SIZEOF_JSAMPARRAY * 4
mov JSAMPARRAY [rsp + 0 * SIZEOF_JSAMPARRAY], rsip ; intpr00
mov JSAMPARRAY [rsp + 1 * SIZEOF_JSAMPARRAY], rbxp ; intpr1
mov JSAMPARRAY [rsp + 2 * SIZEOF_JSAMPARRAY], rdxp ; intpr2
mov rbx, rsp
push rdi
push rcx
push rax
%ifdef WIN64
mov r8, rcx
mov r9, rdi
mov rcx, rax
mov rdx, rbx
%else
mov rdx, rcx
mov rcx, rdi
mov rdi, rax
mov rsi, rbx
%endif
call EXTN(jsimd_h2v1_merged_upsample_sse2)
pop rax
pop rcx
pop rdi
mov rsip, JSAMPARRAY [rsp + 0 * SIZEOF_JSAMPARRAY]
mov rbxp, JSAMPARRAY [rsp + 1 * SIZEOF_JSAMPARRAY]
mov rdxp, JSAMPARRAY [rsp + 2 * SIZEOF_JSAMPARRAY]
add rdi, byte SIZEOF_JSAMPROW ; outptr1
add rsi, byte SIZEOF_JSAMPROW ; inptr01
mov JSAMPARRAY [rsp + 0 * SIZEOF_JSAMPARRAY], rsip ; intpr00
mov JSAMPARRAY [rsp + 1 * SIZEOF_JSAMPARRAY], rbxp ; intpr1
mov JSAMPARRAY [rsp + 2 * SIZEOF_JSAMPARRAY], rdxp ; intpr2
mov rbx, rsp
push rdi
push rcx
push rax
%ifdef WIN64
mov r8, rcx
mov r9, rdi
mov rcx, rax
mov rdx, rbx
%else
mov rdx, rcx
mov rcx, rdi
mov rdi, rax
mov rsi, rbx
%endif
call EXTN(jsimd_h2v1_merged_upsample_sse2)
pop rax
pop rcx
pop rdi
mov rsip, JSAMPARRAY [rsp + 0 * SIZEOF_JSAMPARRAY]
mov rbxp, JSAMPARRAY [rsp + 1 * SIZEOF_JSAMPARRAY]
mov rdxp, JSAMPARRAY [rsp + 2 * SIZEOF_JSAMPARRAY]
add rsp, SIZEOF_JSAMPARRAY * 4
pop rbx
UNCOLLECT_ARGS 4
pop rbp
ret
; For some reason, the OS X linker does not honor the request to align the
; segment unless we do this.
align 32
lib/libjpeg-turbo/simd/x86_64/jdsample-sse2.asm
Executable
+663
View File
@@ -0,0 +1,663 @@
;
; Upsampling (64-bit SSE2)
;
; Copyright 2009 Pierre Ossman <[email protected]> for Cendio AB
; Copyright (C) 2009, 2016, 2024-2026, D. R. Commander.
; Copyright (C) 2018, Matthias Räncker.
; Copyright (C) 2023, Aliaksiej Kandracienka.
;
; Based on the x86 SIMD extension for IJG JPEG library
; Copyright (C) 1999-2006, MIYASAKA Masaru.
; For conditions of distribution and use, see copyright notice in jsimdext.inc
;
; This file should be assembled with NASM (Netwide Assembler) or Yasm.
%include "jsimdext.inc"
; --------------------------------------------------------------------------
SECTION SEG_CONST
ALIGNZ 32
GLOBAL_DATA(jconst_fancy_upsample_sse2)
EXTN(jconst_fancy_upsample_sse2):
PW_ONE times 8 dw 1
PW_TWO times 8 dw 2
PW_THREE times 8 dw 3
PW_SEVEN times 8 dw 7
PW_EIGHT times 8 dw 8
ALIGNZ 32
; --------------------------------------------------------------------------
SECTION SEG_TEXT
BITS 64
; Fancy processing for the common case of 2:1 horizontal and 1:1 vertical.
;
; The upsampling algorithm is linear interpolation between component centers,
; also known as a "triangle filter". This is a good compromise between speed
; and visual quality. The centers of the output components are 1/4 and 3/4 of
; the way between input component centers.
;
; GLOBAL(void)
; jsimd_h2v1_fancy_upsample_sse2(int max_v_samp_factor,
; JDIMENSION downsampled_width,
; JSAMPARRAY input_data,
; JSAMPARRAY *output_data_ptr)
;
; r10 = int max_v_samp_factor
; r11d = JDIMENSION downsampled_width
; r12 = JSAMPARRAY input_data
; r13 = JSAMPARRAY *output_data_ptr
align 32
GLOBAL_FUNCTION(jsimd_h2v1_fancy_upsample_sse2)
EXTN(jsimd_h2v1_fancy_upsample_sse2):
ENDBR64
push rbp
mov rbp, rsp
COLLECT_ARGS 4
mov eax, r11d ; colctr
test rax, rax
jz near .return
mov rcx, r10 ; rowctr
test rcx, rcx
jz near .return
mov rsi, r12 ; input_data
mov rdi, r13
mov rdip, JSAMPARRAY [rdi] ; output_data
.rowloop:
push rax ; colctr
push rdi
push rsi
mov rsip, JSAMPROW [rsi] ; inptr
mov rdip, JSAMPROW [rdi] ; outptr
test rax, SIZEOF_XMMWORD - 1
jz short .skip
mov dl, JSAMPLE [rsi + (rax - 1) * SIZEOF_JSAMPLE]
mov JSAMPLE [rsi + rax * SIZEOF_JSAMPLE], dl
; insert a dummy sample
.skip:
pxor xmm0, xmm0 ; xmm0 = (all 0's)
pcmpeqb xmm7, xmm7
psrldq xmm7, (SIZEOF_XMMWORD - 1)
pand xmm7, XMMWORD [rsi + 0 * SIZEOF_XMMWORD]
add rax, byte SIZEOF_XMMWORD - 1
and rax, byte -SIZEOF_XMMWORD
cmp rax, byte SIZEOF_XMMWORD
ja short .columnloop
.columnloop_last:
pcmpeqb xmm6, xmm6
pslldq xmm6, (SIZEOF_XMMWORD - 1)
pand xmm6, XMMWORD [rsi + 0 * SIZEOF_XMMWORD]
jmp short .upsample
.columnloop:
movdqa xmm6, XMMWORD [rsi + 1 * SIZEOF_XMMWORD]
pslldq xmm6, (SIZEOF_XMMWORD - 1)
.upsample:
movdqa xmm1, XMMWORD [rsi + 0 * SIZEOF_XMMWORD]
movdqa xmm2, xmm1
movdqa xmm3, xmm1 ; xmm1 = ( 0 1 2 ... 13 14 15)
pslldq xmm2, 1 ; xmm2 = (-- 0 1 ... 12 13 14)
psrldq xmm3, 1 ; xmm3 = ( 1 2 3 ... 14 15 --)
por xmm2, xmm7 ; xmm2 = (-1 0 1 ... 12 13 14)
por xmm3, xmm6 ; xmm3 = ( 1 2 3 ... 14 15 16)
movdqa xmm7, xmm1
psrldq xmm7, (SIZEOF_XMMWORD - 1) ; xmm7 = (15 -- -- ... -- -- --)
movdqa xmm4, xmm1
punpcklbw xmm1, xmm0 ; xmm1 = ( 0 1 2 3 4 5 6 7)
punpckhbw xmm4, xmm0 ; xmm4 = ( 8 9 10 11 12 13 14 15)
movdqa xmm5, xmm2
punpcklbw xmm2, xmm0 ; xmm2 = (-1 0 1 2 3 4 5 6)
punpckhbw xmm5, xmm0 ; xmm5 = ( 7 8 9 10 11 12 13 14)
movdqa xmm6, xmm3
punpcklbw xmm3, xmm0 ; xmm3 = ( 1 2 3 4 5 6 7 8)
punpckhbw xmm6, xmm0 ; xmm6 = ( 9 10 11 12 13 14 15 16)
pmullw xmm1, [rel PW_THREE]
pmullw xmm4, [rel PW_THREE]
paddw xmm2, [rel PW_ONE]
paddw xmm5, [rel PW_ONE]
paddw xmm3, [rel PW_TWO]
paddw xmm6, [rel PW_TWO]
paddw xmm2, xmm1
paddw xmm5, xmm4
psrlw xmm2, 2 ; xmm2 = OutLE = ( 0 2 4 6 8 10 12 14)
psrlw xmm5, 2 ; xmm5 = OutHE = (16 18 20 22 24 26 28 30)
paddw xmm3, xmm1
paddw xmm6, xmm4
psrlw xmm3, 2 ; xmm3 = OutLO = ( 1 3 5 7 9 11 13 15)
psrlw xmm6, 2 ; xmm6 = OutHO = (17 19 21 23 25 27 29 31)
psllw xmm3, BYTE_BIT
psllw xmm6, BYTE_BIT
por xmm2, xmm3 ; xmm2 = OutL = ( 0 1 2 ... 13 14 15)
por xmm5, xmm6 ; xmm5 = OutH = (16 17 18 ... 29 30 31)
movdqa XMMWORD [rdi + 0 * SIZEOF_XMMWORD], xmm2
movdqa XMMWORD [rdi + 1 * SIZEOF_XMMWORD], xmm5
sub rax, byte SIZEOF_XMMWORD
add rsi, byte 1 * SIZEOF_XMMWORD ; inptr
add rdi, byte 2 * SIZEOF_XMMWORD ; outptr
cmp rax, byte SIZEOF_XMMWORD
ja near .columnloop
test eax, eax
jnz near .columnloop_last
pop rsi
pop rdi
pop rax
add rsi, byte SIZEOF_JSAMPROW ; input_data
add rdi, byte SIZEOF_JSAMPROW ; output_data
dec rcx ; rowctr
jg near .rowloop
.return:
UNCOLLECT_ARGS 4
pop rbp
ret
; --------------------------------------------------------------------------
;
; Fancy processing for the common case of 2:1 horizontal and 2:1 vertical.
; Again a triangle filter; see comments for h2v1 case, above.
;
; GLOBAL(void)
; jsimd_h2v2_fancy_upsample_sse2(int max_v_samp_factor,
; JDIMENSION downsampled_width,
; JSAMPARRAY input_data,
; JSAMPARRAY *output_data_ptr)
;
; r10 = int max_v_samp_factor
; r11d = JDIMENSION downsampled_width
; r12 = JSAMPARRAY input_data
; r13 = JSAMPARRAY *output_data_ptr
%define wk(i) r15 - (WK_NUM - (i)) * SIZEOF_XMMWORD ; xmmword wk[WK_NUM]
%define WK_NUM 4
align 32
GLOBAL_FUNCTION(jsimd_h2v2_fancy_upsample_sse2)
EXTN(jsimd_h2v2_fancy_upsample_sse2):
ENDBR64
push rbp
mov rbp, rsp
push r15
and rsp, byte (-SIZEOF_XMMWORD) ; align to 128 bits
; Allocate stack space for wk array. r15 is used to access it.
mov r15, rsp
sub rsp, byte (SIZEOF_XMMWORD * WK_NUM)
COLLECT_ARGS 4
push rbx
mov eax, r11d ; colctr
test rax, rax
jz near .return
mov rcx, r10 ; rowctr
test rcx, rcx
jz near .return
mov rsi, r12 ; input_data
mov rdi, r13
mov rdip, JSAMPARRAY [rdi] ; output_data
.rowloop:
push rax ; colctr
push rcx
push rdi
push rsi
mov rcxp, JSAMPROW [rsi - 1 * SIZEOF_JSAMPROW] ; inptr1(above)
mov rbxp, JSAMPROW [rsi + 0 * SIZEOF_JSAMPROW] ; inptr0
mov rsip, JSAMPROW [rsi + 1 * SIZEOF_JSAMPROW] ; inptr1(below)
mov rdxp, JSAMPROW [rdi + 0 * SIZEOF_JSAMPROW] ; outptr0
mov rdip, JSAMPROW [rdi + 1 * SIZEOF_JSAMPROW] ; outptr1
test rax, SIZEOF_XMMWORD - 1
jz short .skip
push rdx
mov dl, JSAMPLE [rcx + (rax - 1) * SIZEOF_JSAMPLE]
mov JSAMPLE [rcx + rax * SIZEOF_JSAMPLE], dl
mov dl, JSAMPLE [rbx + (rax - 1) * SIZEOF_JSAMPLE]
mov JSAMPLE [rbx + rax * SIZEOF_JSAMPLE], dl
mov dl, JSAMPLE [rsi + (rax - 1) * SIZEOF_JSAMPLE]
mov JSAMPLE [rsi + rax * SIZEOF_JSAMPLE], dl
; insert a dummy sample
pop rdx
.skip:
; -- process the first column block
movdqa xmm0, XMMWORD [rbx + 0 * SIZEOF_XMMWORD] ; xmm0 = row[ 0][0]
movdqa xmm1, XMMWORD [rcx + 0 * SIZEOF_XMMWORD] ; xmm1 = row[-1][0]
movdqa xmm2, XMMWORD [rsi + 0 * SIZEOF_XMMWORD] ; xmm2 = row[+1][0]
pxor xmm3, xmm3 ; xmm3 = (all 0's)
movdqa xmm4, xmm0
punpcklbw xmm0, xmm3 ; xmm0 = row[ 0]( 0 1 2 3 4 5 6 7)
punpckhbw xmm4, xmm3 ; xmm4 = row[ 0]( 8 9 10 11 12 13 14 15)
movdqa xmm5, xmm1
punpcklbw xmm1, xmm3 ; xmm1 = row[-1]( 0 1 2 3 4 5 6 7)
punpckhbw xmm5, xmm3 ; xmm5 = row[-1]( 8 9 10 11 12 13 14 15)
movdqa xmm6, xmm2
punpcklbw xmm2, xmm3 ; xmm2 = row[+1]( 0 1 2 3 4 5 6 7)
punpckhbw xmm6, xmm3 ; xmm6 = row[+1]( 8 9 10 11 12 13 14 15)
pmullw xmm0, [rel PW_THREE]
pmullw xmm4, [rel PW_THREE]
pcmpeqb xmm7, xmm7
psrldq xmm7, (SIZEOF_XMMWORD - 2)
paddw xmm1, xmm0 ; xmm1 = Int0L = ( 0 1 2 3 4 5 6 7)
paddw xmm5, xmm4 ; xmm5 = Int0H = ( 8 9 10 11 12 13 14 15)
paddw xmm2, xmm0 ; xmm2 = Int1L = ( 0 1 2 3 4 5 6 7)
paddw xmm6, xmm4 ; xmm6 = Int1H = ( 8 9 10 11 12 13 14 15)
; temporarily save the intermediate data
movdqa XMMWORD [rdx + 0 * SIZEOF_XMMWORD], xmm1
movdqa XMMWORD [rdx + 1 * SIZEOF_XMMWORD], xmm5
movdqa XMMWORD [rdi + 0 * SIZEOF_XMMWORD], xmm2
movdqa XMMWORD [rdi + 1 * SIZEOF_XMMWORD], xmm6
pand xmm1, xmm7 ; xmm1 = ( 0 -- -- -- -- -- -- --)
pand xmm2, xmm7 ; xmm2 = ( 0 -- -- -- -- -- -- --)
movdqa XMMWORD [wk(0)], xmm1
movdqa XMMWORD [wk(1)], xmm2
add rax, byte SIZEOF_XMMWORD - 1
and rax, byte -SIZEOF_XMMWORD
cmp rax, byte SIZEOF_XMMWORD
ja short .columnloop
.columnloop_last:
; -- process the last column block
pcmpeqb xmm1, xmm1
pslldq xmm1, (SIZEOF_XMMWORD - 2)
movdqa xmm2, xmm1
pand xmm1, XMMWORD [rdx + 1 * SIZEOF_XMMWORD]
pand xmm2, XMMWORD [rdi + 1 * SIZEOF_XMMWORD]
movdqa XMMWORD [wk(2)], xmm1 ; xmm1 = (-- -- -- -- -- -- -- 15)
movdqa XMMWORD [wk(3)], xmm2 ; xmm2 = (-- -- -- -- -- -- -- 15)
jmp near .upsample
.columnloop:
; -- process the next column block
movdqa xmm0, XMMWORD [rbx + 1 * SIZEOF_XMMWORD] ; xmm0 = row[ 0][1]
movdqa xmm1, XMMWORD [rcx + 1 * SIZEOF_XMMWORD] ; xmm1 = row[-1][1]
movdqa xmm2, XMMWORD [rsi + 1 * SIZEOF_XMMWORD] ; xmm2 = row[+1][1]
pxor xmm3, xmm3 ; xmm3 = (all 0's)
movdqa xmm4, xmm0
punpcklbw xmm0, xmm3 ; xmm0 = row[ 0]( 0 1 2 3 4 5 6 7)
punpckhbw xmm4, xmm3 ; xmm4 = row[ 0]( 8 9 10 11 12 13 14 15)
movdqa xmm5, xmm1
punpcklbw xmm1, xmm3 ; xmm1 = row[-1]( 0 1 2 3 4 5 6 7)
punpckhbw xmm5, xmm3 ; xmm5 = row[-1]( 8 9 10 11 12 13 14 15)
movdqa xmm6, xmm2
punpcklbw xmm2, xmm3 ; xmm2 = row[+1]( 0 1 2 3 4 5 6 7)
punpckhbw xmm6, xmm3 ; xmm6 = row[+1]( 8 9 10 11 12 13 14 15)
pmullw xmm0, [rel PW_THREE]
pmullw xmm4, [rel PW_THREE]
paddw xmm1, xmm0 ; xmm1 = Int0L = ( 0 1 2 3 4 5 6 7)
paddw xmm5, xmm4 ; xmm5 = Int0H = ( 8 9 10 11 12 13 14 15)
paddw xmm2, xmm0 ; xmm2 = Int1L = ( 0 1 2 3 4 5 6 7)
paddw xmm6, xmm4 ; xmm6 = Int1H = ( 8 9 10 11 12 13 14 15)
; temporarily save the intermediate data
movdqa XMMWORD [rdx + 2 * SIZEOF_XMMWORD], xmm1
movdqa XMMWORD [rdx + 3 * SIZEOF_XMMWORD], xmm5
movdqa XMMWORD [rdi + 2 * SIZEOF_XMMWORD], xmm2
movdqa XMMWORD [rdi + 3 * SIZEOF_XMMWORD], xmm6
pslldq xmm1, (SIZEOF_XMMWORD - 2) ; xmm1 = (-- -- -- -- -- -- -- 0)
pslldq xmm2, (SIZEOF_XMMWORD - 2) ; xmm2 = (-- -- -- -- -- -- -- 0)
movdqa XMMWORD [wk(2)], xmm1
movdqa XMMWORD [wk(3)], xmm2
.upsample:
; -- process the upper row
movdqa xmm7, XMMWORD [rdx + 0 * SIZEOF_XMMWORD]
movdqa xmm3, XMMWORD [rdx + 1 * SIZEOF_XMMWORD]
movdqa xmm0, xmm7 ; xmm7 = Int0L = ( 0 1 2 3 4 5 6 7)
movdqa xmm4, xmm3 ; xmm3 = Int0H = ( 8 9 10 11 12 13 14 15)
psrldq xmm0, 2 ; xmm0 = ( 1 2 3 4 5 6 7 --)
pslldq xmm4, (SIZEOF_XMMWORD - 2) ; xmm4 = (-- -- -- -- -- -- -- 8)
movdqa xmm5, xmm7
movdqa xmm6, xmm3
psrldq xmm5, (SIZEOF_XMMWORD - 2) ; xmm5 = ( 7 -- -- -- -- -- -- --)
pslldq xmm6, 2 ; xmm6 = (-- 8 9 10 11 12 13 14)
por xmm0, xmm4 ; xmm0 = ( 1 2 3 4 5 6 7 8)
por xmm5, xmm6 ; xmm5 = ( 7 8 9 10 11 12 13 14)
movdqa xmm1, xmm7
movdqa xmm2, xmm3
pslldq xmm1, 2 ; xmm1 = (-- 0 1 2 3 4 5 6)
psrldq xmm2, 2 ; xmm2 = ( 9 10 11 12 13 14 15 --)
movdqa xmm4, xmm3
psrldq xmm4, (SIZEOF_XMMWORD - 2) ; xmm4 = (15 -- -- -- -- -- -- --)
por xmm1, XMMWORD [wk(0)] ; xmm1 = (-1 0 1 2 3 4 5 6)
por xmm2, XMMWORD [wk(2)] ; xmm2 = ( 9 10 11 12 13 14 15 16)
movdqa XMMWORD [wk(0)], xmm4
pmullw xmm7, [rel PW_THREE]
pmullw xmm3, [rel PW_THREE]
paddw xmm1, [rel PW_EIGHT]
paddw xmm5, [rel PW_EIGHT]
paddw xmm0, [rel PW_SEVEN]
paddw xmm2, [rel PW_SEVEN]
paddw xmm1, xmm7
paddw xmm5, xmm3
psrlw xmm1, 4 ; xmm1 = Out0LE = ( 0 2 4 6 8 10 12 14)
psrlw xmm5, 4 ; xmm5 = Out0HE = (16 18 20 22 24 26 28 30)
paddw xmm0, xmm7
paddw xmm2, xmm3
psrlw xmm0, 4 ; xmm0 = Out0LO = ( 1 3 5 7 9 11 13 15)
psrlw xmm2, 4 ; xmm2 = Out0HO = (17 19 21 23 25 27 29 31)
psllw xmm0, BYTE_BIT
psllw xmm2, BYTE_BIT
por xmm1, xmm0 ; xmm1 = Out0L = ( 0 1 2 ... 13 14 15)
por xmm5, xmm2 ; xmm5 = Out0H = (16 17 18 ... 29 30 31)
movdqa XMMWORD [rdx + 0 * SIZEOF_XMMWORD], xmm1
movdqa XMMWORD [rdx + 1 * SIZEOF_XMMWORD], xmm5
; -- process the lower row
movdqa xmm6, XMMWORD [rdi + 0 * SIZEOF_XMMWORD]
movdqa xmm4, XMMWORD [rdi + 1 * SIZEOF_XMMWORD]
movdqa xmm7, xmm6 ; xmm6 = Int1L = ( 0 1 2 3 4 5 6 7)
movdqa xmm3, xmm4 ; xmm4 = Int1H = ( 8 9 10 11 12 13 14 15)
psrldq xmm7, 2 ; xmm7 = ( 1 2 3 4 5 6 7 --)
pslldq xmm3, (SIZEOF_XMMWORD - 2) ; xmm3 = (-- -- -- -- -- -- -- 8)
movdqa xmm0, xmm6
movdqa xmm2, xmm4
psrldq xmm0, (SIZEOF_XMMWORD - 2) ; xmm0 = ( 7 -- -- -- -- -- -- --)
pslldq xmm2, 2 ; xmm2 = (-- 8 9 10 11 12 13 14)
por xmm7, xmm3 ; xmm7 = ( 1 2 3 4 5 6 7 8)
por xmm0, xmm2 ; xmm0 = ( 7 8 9 10 11 12 13 14)
movdqa xmm1, xmm6
movdqa xmm5, xmm4
pslldq xmm1, 2 ; xmm1 = (-- 0 1 2 3 4 5 6)
psrldq xmm5, 2 ; xmm5 = ( 9 10 11 12 13 14 15 --)
movdqa xmm3, xmm4
psrldq xmm3, (SIZEOF_XMMWORD - 2) ; xmm3 = (15 -- -- -- -- -- -- --)
por xmm1, XMMWORD [wk(1)] ; xmm1 = (-1 0 1 2 3 4 5 6)
por xmm5, XMMWORD [wk(3)] ; xmm5 = ( 9 10 11 12 13 14 15 16)
movdqa XMMWORD [wk(1)], xmm3
pmullw xmm6, [rel PW_THREE]
pmullw xmm4, [rel PW_THREE]
paddw xmm1, [rel PW_EIGHT]
paddw xmm0, [rel PW_EIGHT]
paddw xmm7, [rel PW_SEVEN]
paddw xmm5, [rel PW_SEVEN]
paddw xmm1, xmm6
paddw xmm0, xmm4
psrlw xmm1, 4 ; xmm1 = Out1LE = ( 0 2 4 6 8 10 12 14)
psrlw xmm0, 4 ; xmm0 = Out1HE = (16 18 20 22 24 26 28 30)
paddw xmm7, xmm6
paddw xmm5, xmm4
psrlw xmm7, 4 ; xmm7 = Out1LO = ( 1 3 5 7 9 11 13 15)
psrlw xmm5, 4 ; xmm5 = Out1HO = (17 19 21 23 25 27 29 31)
psllw xmm7, BYTE_BIT
psllw xmm5, BYTE_BIT
por xmm1, xmm7 ; xmm1 = Out1L = ( 0 1 2 ... 13 14 15)
por xmm0, xmm5 ; xmm0 = Out1H = (16 17 18 ... 29 30 31)
movdqa XMMWORD [rdi + 0 * SIZEOF_XMMWORD], xmm1
movdqa XMMWORD [rdi + 1 * SIZEOF_XMMWORD], xmm0
sub rax, byte SIZEOF_XMMWORD
add rcx, byte 1 * SIZEOF_XMMWORD ; inptr1(above)
add rbx, byte 1 * SIZEOF_XMMWORD ; inptr0
add rsi, byte 1 * SIZEOF_XMMWORD ; inptr1(below)
add rdx, byte 2 * SIZEOF_XMMWORD ; outptr0
add rdi, byte 2 * SIZEOF_XMMWORD ; outptr1
cmp rax, byte SIZEOF_XMMWORD
ja near .columnloop
test rax, rax
jnz near .columnloop_last
pop rsi
pop rdi
pop rcx
pop rax
add rsi, byte 1 * SIZEOF_JSAMPROW ; input_data
add rdi, byte 2 * SIZEOF_JSAMPROW ; output_data
sub rcx, byte 2 ; rowctr
jg near .rowloop
.return:
pop rbx
UNCOLLECT_ARGS 4
lea rsp, [rbp - 8]
pop r15
pop rbp
ret
; --------------------------------------------------------------------------
;
; Fast processing for the common case of 2:1 horizontal and 1:1 vertical.
; It's still a box filter.
;
; GLOBAL(void)
; jsimd_h2v1_upsample_sse2(int max_v_samp_factor, JDIMENSION output_width,
; JSAMPARRAY input_data, JSAMPARRAY *output_data_ptr)
;
; r10 = int max_v_samp_factor
; r11d = JDIMENSION output_width
; r12 = JSAMPARRAY input_data
; r13 = JSAMPARRAY *output_data_ptr
align 32
GLOBAL_FUNCTION(jsimd_h2v1_upsample_sse2)
EXTN(jsimd_h2v1_upsample_sse2):
ENDBR64
push rbp
mov rbp, rsp
COLLECT_ARGS 4
mov edx, r11d
add rdx, byte (2 * SIZEOF_XMMWORD) - 1
and rdx, byte -(2 * SIZEOF_XMMWORD)
jz near .return
mov rcx, r10 ; rowctr
test rcx, rcx
jz short .return
mov rsi, r12 ; input_data
mov rdi, r13
mov rdip, JSAMPARRAY [rdi] ; output_data
.rowloop:
push rdi
push rsi
mov rsip, JSAMPROW [rsi] ; inptr
mov rdip, JSAMPROW [rdi] ; outptr
mov rax, rdx ; colctr
.columnloop:
movdqa xmm0, XMMWORD [rsi + 0 * SIZEOF_XMMWORD]
movdqa xmm1, xmm0
punpcklbw xmm0, xmm0
punpckhbw xmm1, xmm1
movdqa XMMWORD [rdi + 0 * SIZEOF_XMMWORD], xmm0
movdqa XMMWORD [rdi + 1 * SIZEOF_XMMWORD], xmm1
sub rax, byte 2 * SIZEOF_XMMWORD
jz short .nextrow
movdqa xmm2, XMMWORD [rsi + 1 * SIZEOF_XMMWORD]
movdqa xmm3, xmm2
punpcklbw xmm2, xmm2
punpckhbw xmm3, xmm3
movdqa XMMWORD [rdi + 2 * SIZEOF_XMMWORD], xmm2
movdqa XMMWORD [rdi + 3 * SIZEOF_XMMWORD], xmm3
sub rax, byte 2 * SIZEOF_XMMWORD
jz short .nextrow
add rsi, byte 2 * SIZEOF_XMMWORD ; inptr
add rdi, byte 4 * SIZEOF_XMMWORD ; outptr
jmp short .columnloop
.nextrow:
pop rsi
pop rdi
add rsi, byte SIZEOF_JSAMPROW ; input_data
add rdi, byte SIZEOF_JSAMPROW ; output_data
dec rcx ; rowctr
jg short .rowloop
.return:
UNCOLLECT_ARGS 4
pop rbp
ret
; --------------------------------------------------------------------------
;
; Fast processing for the common case of 2:1 horizontal and 2:1 vertical.
; It's still a box filter.
;
; GLOBAL(void)
; jsimd_h2v2_upsample_sse2(int max_v_samp_factor, JDIMENSION output_width,
; JSAMPARRAY input_data, JSAMPARRAY *output_data_ptr)
;
; r10 = int max_v_samp_factor
; r11d = JDIMENSION output_width
; r12 = JSAMPARRAY input_data
; r13 = JSAMPARRAY *output_data_ptr
align 32
GLOBAL_FUNCTION(jsimd_h2v2_upsample_sse2)
EXTN(jsimd_h2v2_upsample_sse2):
ENDBR64
push rbp
mov rbp, rsp
COLLECT_ARGS 4
push rbx
mov edx, r11d
add rdx, byte (2 * SIZEOF_XMMWORD) - 1
and rdx, byte -(2 * SIZEOF_XMMWORD)
jz near .return
mov rcx, r10 ; rowctr
test rcx, rcx
jz near .return
mov rsi, r12 ; input_data
mov rdi, r13
mov rdip, JSAMPARRAY [rdi] ; output_data
.rowloop:
push rdi
push rsi
mov rsip, JSAMPROW [rsi] ; inptr
mov rbxp, JSAMPROW [rdi + 0 * SIZEOF_JSAMPROW] ; outptr0
mov rdip, JSAMPROW [rdi + 1 * SIZEOF_JSAMPROW] ; outptr1
mov rax, rdx ; colctr
.columnloop:
movdqa xmm0, XMMWORD [rsi + 0 * SIZEOF_XMMWORD]
movdqa xmm1, xmm0
punpcklbw xmm0, xmm0
punpckhbw xmm1, xmm1
movdqa XMMWORD [rbx + 0 * SIZEOF_XMMWORD], xmm0
movdqa XMMWORD [rbx + 1 * SIZEOF_XMMWORD], xmm1
movdqa XMMWORD [rdi + 0 * SIZEOF_XMMWORD], xmm0
movdqa XMMWORD [rdi + 1 * SIZEOF_XMMWORD], xmm1
sub rax, byte 2 * SIZEOF_XMMWORD
jz short .nextrow
movdqa xmm2, XMMWORD [rsi + 1 * SIZEOF_XMMWORD]
movdqa xmm3, xmm2
punpcklbw xmm2, xmm2
punpckhbw xmm3, xmm3
movdqa XMMWORD [rbx + 2 * SIZEOF_XMMWORD], xmm2
movdqa XMMWORD [rbx + 3 * SIZEOF_XMMWORD], xmm3
movdqa XMMWORD [rdi + 2 * SIZEOF_XMMWORD], xmm2
movdqa XMMWORD [rdi + 3 * SIZEOF_XMMWORD], xmm3
sub rax, byte 2 * SIZEOF_XMMWORD
jz short .nextrow
add rsi, byte 2 * SIZEOF_XMMWORD ; inptr
add rbx, byte 4 * SIZEOF_XMMWORD ; outptr0
add rdi, byte 4 * SIZEOF_XMMWORD ; outptr1
jmp short .columnloop
.nextrow:
pop rsi
pop rdi
add rsi, byte 1 * SIZEOF_JSAMPROW ; input_data
add rdi, byte 2 * SIZEOF_JSAMPROW ; output_data
sub rcx, byte 2 ; rowctr
jg near .rowloop
.return:
pop rbx
UNCOLLECT_ARGS 4
pop rbp
ret
; For some reason, the OS X linker does not honor the request to align the
; segment unless we do this.
align 32
lib/libjpeg-turbo/simd/x86_64/jfdctflt-sse.asm
Executable
+360
View File
@@ -0,0 +1,360 @@
;
; Floating-point FDCT (64-bit SSE)
;
; Copyright 2009 Pierre Ossman <[email protected]> for Cendio AB
; Copyright (C) 2009, 2016, 2024-2025, D. R. Commander.
; Copyright (C) 2023, Aliaksiej Kandracienka.
;
; Based on the x86 SIMD extension for IJG JPEG library
; Copyright (C) 1999-2006, MIYASAKA Masaru.
; For conditions of distribution and use, see copyright notice in jsimdext.inc
;
; This file should be assembled with NASM (Netwide Assembler) or Yasm.
;
; This file contains a floating-point implementation of the forward DCT
; (Discrete Cosine Transform). The following code is based directly on the
; IJG's original jfdctflt.c; see jfdctflt.c for more details.
%include "jsimdext.inc"
%include "jdct.inc"
; --------------------------------------------------------------------------
%macro unpcklps2 2 ; %1 = (0 1 2 3) / %2 = (4 5 6 7) => %1 = (0 1 4 5)
shufps %1, %2, 0x44
%endmacro
%macro unpckhps2 2 ; %1 = (0 1 2 3) / %2 = (4 5 6 7) => %1 = (2 3 6 7)
shufps %1, %2, 0xEE
%endmacro
; --------------------------------------------------------------------------
SECTION SEG_CONST
ALIGNZ 32
GLOBAL_DATA(jconst_fdct_float_sse)
EXTN(jconst_fdct_float_sse):
PD_0_382 times 4 dd 0.382683432365089771728460
PD_0_707 times 4 dd 0.707106781186547524400844
PD_0_541 times 4 dd 0.541196100146196984399723
PD_1_306 times 4 dd 1.306562964876376527856643
ALIGNZ 32
; --------------------------------------------------------------------------
SECTION SEG_TEXT
BITS 64
; Perform the forward DCT on one block of samples.
;
; GLOBAL(void)
; jsimd_fdct_float_sse(FAST_FLOAT *data)
;
; r10 = FAST_FLOAT *data
%define wk(i) r15 - (WK_NUM - (i)) * SIZEOF_XMMWORD ; xmmword wk[WK_NUM]
%define WK_NUM 2
align 32
GLOBAL_FUNCTION(jsimd_fdct_float_sse)
EXTN(jsimd_fdct_float_sse):
ENDBR64
push rbp
mov rbp, rsp
push r15
and rsp, byte (-SIZEOF_XMMWORD) ; align to 128 bits
; Allocate stack space for wk array. r15 is used to access it.
mov r15, rsp
sub rsp, byte (SIZEOF_XMMWORD * WK_NUM)
COLLECT_ARGS 1
; ---- Pass 1: process rows.
mov rdx, r10 ; (FAST_FLOAT *)
mov rcx, DCTSIZE / 4
.rowloop:
movaps xmm0, XMMWORD [XMMBLOCK(2, 0, rdx, SIZEOF_FAST_FLOAT)]
movaps xmm1, XMMWORD [XMMBLOCK(3, 0, rdx, SIZEOF_FAST_FLOAT)]
movaps xmm2, XMMWORD [XMMBLOCK(2, 1, rdx, SIZEOF_FAST_FLOAT)]
movaps xmm3, XMMWORD [XMMBLOCK(3, 1, rdx, SIZEOF_FAST_FLOAT)]
; xmm0 = (20 21 22 23)
; xmm2 = (24 25 26 27)
; xmm1 = (30 31 32 33)
; xmm3 = (34 35 36 37)
movaps xmm4, xmm0 ; transpose coefficients(phase 1)
unpcklps xmm0, xmm1 ; xmm0 = (20 30 21 31)
unpckhps xmm4, xmm1 ; xmm4 = (22 32 23 33)
movaps xmm5, xmm2 ; transpose coefficients(phase 1)
unpcklps xmm2, xmm3 ; xmm2 = (24 34 25 35)
unpckhps xmm5, xmm3 ; xmm5 = (26 36 27 37)
movaps xmm6, XMMWORD [XMMBLOCK(0, 0, rdx, SIZEOF_FAST_FLOAT)]
movaps xmm7, XMMWORD [XMMBLOCK(1, 0, rdx, SIZEOF_FAST_FLOAT)]
movaps xmm1, XMMWORD [XMMBLOCK(0, 1, rdx, SIZEOF_FAST_FLOAT)]
movaps xmm3, XMMWORD [XMMBLOCK(1, 1, rdx, SIZEOF_FAST_FLOAT)]
; xmm6 = (00 01 02 03)
; xmm1 = (04 05 06 07)
; xmm7 = (10 11 12 13)
; xmm3 = (14 15 16 17)
movaps XMMWORD [wk(0)], xmm4 ; wk(0) = (22 32 23 33)
movaps XMMWORD [wk(1)], xmm2 ; wk(1) = (24 34 25 35)
movaps xmm4, xmm6 ; transpose coefficients(phase 1)
unpcklps xmm6, xmm7 ; xmm6 = (00 10 01 11)
unpckhps xmm4, xmm7 ; xmm4 = (02 12 03 13)
movaps xmm2, xmm1 ; transpose coefficients(phase 1)
unpcklps xmm1, xmm3 ; xmm1 = (04 14 05 15)
unpckhps xmm2, xmm3 ; xmm2 = (06 16 07 17)
movaps xmm7, xmm6 ; transpose coefficients(phase 2)
unpcklps2 xmm6, xmm0 ; xmm6 = (00 10 20 30) = data0
unpckhps2 xmm7, xmm0 ; xmm7 = (01 11 21 31) = data1
movaps xmm3, xmm2 ; transpose coefficients(phase 2)
unpcklps2 xmm2, xmm5 ; xmm2 = (06 16 26 36) = data6
unpckhps2 xmm3, xmm5 ; xmm3 = (07 17 27 37) = data7
movaps xmm0, xmm7
movaps xmm5, xmm6
subps xmm7, xmm2 ; xmm7 = data1 - data6 = tmp6
subps xmm6, xmm3 ; xmm6 = data0 - data7 = tmp7
addps xmm0, xmm2 ; xmm0 = data1 + data6 = tmp1
addps xmm5, xmm3 ; xmm5 = data0 + data7 = tmp0
movaps xmm2, XMMWORD [wk(0)] ; xmm2 = (22 32 23 33)
movaps xmm3, XMMWORD [wk(1)] ; xmm3 = (24 34 25 35)
movaps XMMWORD [wk(0)], xmm7 ; wk(0) = tmp6
movaps XMMWORD [wk(1)], xmm6 ; wk(1) = tmp7
movaps xmm7, xmm4 ; transpose coefficients(phase 2)
unpcklps2 xmm4, xmm2 ; xmm4 = (02 12 22 32) = data2
unpckhps2 xmm7, xmm2 ; xmm7 = (03 13 23 33) = data3
movaps xmm6, xmm1 ; transpose coefficients(phase 2)
unpcklps2 xmm1, xmm3 ; xmm1 = (04 14 24 34) = data4
unpckhps2 xmm6, xmm3 ; xmm6 = (05 15 25 35) = data5
movaps xmm2, xmm7
movaps xmm3, xmm4
addps xmm7, xmm1 ; xmm7 = data3 + data4 = tmp3
addps xmm4, xmm6 ; xmm4 = data2 + data5 = tmp2
subps xmm2, xmm1 ; xmm2 = data3 - data4 = tmp4
subps xmm3, xmm6 ; xmm3 = data2 - data5 = tmp5
; -- Even part
movaps xmm1, xmm5
movaps xmm6, xmm0
subps xmm5, xmm7 ; xmm5 = tmp13
subps xmm0, xmm4 ; xmm0 = tmp12
addps xmm1, xmm7 ; xmm1 = tmp10
addps xmm6, xmm4 ; xmm6 = tmp11
addps xmm0, xmm5
mulps xmm0, [rel PD_0_707] ; xmm0 = z1
movaps xmm7, xmm1
movaps xmm4, xmm5
subps xmm1, xmm6 ; xmm1 = data4
subps xmm5, xmm0 ; xmm5 = data6
addps xmm7, xmm6 ; xmm7 = data0
addps xmm4, xmm0 ; xmm4 = data2
movaps XMMWORD [XMMBLOCK(0, 1, rdx, SIZEOF_FAST_FLOAT)], xmm1
movaps XMMWORD [XMMBLOCK(2, 1, rdx, SIZEOF_FAST_FLOAT)], xmm5
movaps XMMWORD [XMMBLOCK(0, 0, rdx, SIZEOF_FAST_FLOAT)], xmm7
movaps XMMWORD [XMMBLOCK(2, 0, rdx, SIZEOF_FAST_FLOAT)], xmm4
; -- Odd part
movaps xmm6, XMMWORD [wk(0)] ; xmm6 = tmp6
movaps xmm0, XMMWORD [wk(1)] ; xmm0 = tmp7
addps xmm2, xmm3 ; xmm2 = tmp10
addps xmm3, xmm6 ; xmm3 = tmp11
addps xmm6, xmm0 ; xmm6 = tmp12, xmm0 = tmp7
mulps xmm3, [rel PD_0_707] ; xmm3 = z3
movaps xmm1, xmm2 ; xmm1 = tmp10
subps xmm2, xmm6
mulps xmm2, [rel PD_0_382] ; xmm2 = z5
mulps xmm1, [rel PD_0_541] ; xmm1 = MULTIPLY(tmp10, FIX_0_541196)
mulps xmm6, [rel PD_1_306] ; xmm6 = MULTIPLY(tmp12, FIX_1_306562)
addps xmm1, xmm2 ; xmm1 = z2
addps xmm6, xmm2 ; xmm6 = z4
movaps xmm5, xmm0
subps xmm0, xmm3 ; xmm0 = z13
addps xmm5, xmm3 ; xmm5 = z11
movaps xmm7, xmm0
movaps xmm4, xmm5
subps xmm0, xmm1 ; xmm0 = data3
subps xmm5, xmm6 ; xmm5 = data7
addps xmm7, xmm1 ; xmm7 = data5
addps xmm4, xmm6 ; xmm4 = data1
movaps XMMWORD [XMMBLOCK(3, 0, rdx, SIZEOF_FAST_FLOAT)], xmm0
movaps XMMWORD [XMMBLOCK(3, 1, rdx, SIZEOF_FAST_FLOAT)], xmm5
movaps XMMWORD [XMMBLOCK(1, 1, rdx, SIZEOF_FAST_FLOAT)], xmm7
movaps XMMWORD [XMMBLOCK(1, 0, rdx, SIZEOF_FAST_FLOAT)], xmm4
add rdx, 4 * DCTSIZE * SIZEOF_FAST_FLOAT
dec rcx
jnz near .rowloop
; ---- Pass 2: process columns.
mov rdx, r10 ; (FAST_FLOAT *)
mov rcx, DCTSIZE / 4
.columnloop:
movaps xmm0, XMMWORD [XMMBLOCK(2, 0, rdx, SIZEOF_FAST_FLOAT)]
movaps xmm1, XMMWORD [XMMBLOCK(3, 0, rdx, SIZEOF_FAST_FLOAT)]
movaps xmm2, XMMWORD [XMMBLOCK(6, 0, rdx, SIZEOF_FAST_FLOAT)]
movaps xmm3, XMMWORD [XMMBLOCK(7, 0, rdx, SIZEOF_FAST_FLOAT)]
; xmm0 = (02 12 22 32)
; xmm2 = (42 52 62 72)
; xmm1 = (03 13 23 33)
; xmm3 = (43 53 63 73)
movaps xmm4, xmm0 ; transpose coefficients(phase 1)
unpcklps xmm0, xmm1 ; xmm0 = (02 03 12 13)
unpckhps xmm4, xmm1 ; xmm4 = (22 23 32 33)
movaps xmm5, xmm2 ; transpose coefficients(phase 1)
unpcklps xmm2, xmm3 ; xmm2 = (42 43 52 53)
unpckhps xmm5, xmm3 ; xmm5 = (62 63 72 73)
movaps xmm6, XMMWORD [XMMBLOCK(0, 0, rdx, SIZEOF_FAST_FLOAT)]
movaps xmm7, XMMWORD [XMMBLOCK(1, 0, rdx, SIZEOF_FAST_FLOAT)]
movaps xmm1, XMMWORD [XMMBLOCK(4, 0, rdx, SIZEOF_FAST_FLOAT)]
movaps xmm3, XMMWORD [XMMBLOCK(5, 0, rdx, SIZEOF_FAST_FLOAT)]
; xmm6 = (00 10 20 30)
; xmm1 = (40 50 60 70)
; xmm7 = (01 11 21 31)
; xmm3 = (41 51 61 71)
movaps XMMWORD [wk(0)], xmm4 ; wk(0) = (22 23 32 33)
movaps XMMWORD [wk(1)], xmm2 ; wk(1) = (42 43 52 53)
movaps xmm4, xmm6 ; transpose coefficients(phase 1)
unpcklps xmm6, xmm7 ; xmm6 = (00 01 10 11)
unpckhps xmm4, xmm7 ; xmm4 = (20 21 30 31)
movaps xmm2, xmm1 ; transpose coefficients(phase 1)
unpcklps xmm1, xmm3 ; xmm1 = (40 41 50 51)
unpckhps xmm2, xmm3 ; xmm2 = (60 61 70 71)
movaps xmm7, xmm6 ; transpose coefficients(phase 2)
unpcklps2 xmm6, xmm0 ; xmm6 = (00 01 02 03) = data0
unpckhps2 xmm7, xmm0 ; xmm7 = (10 11 12 13) = data1
movaps xmm3, xmm2 ; transpose coefficients(phase 2)
unpcklps2 xmm2, xmm5 ; xmm2 = (60 61 62 63) = data6
unpckhps2 xmm3, xmm5 ; xmm3 = (70 71 72 73) = data7
movaps xmm0, xmm7
movaps xmm5, xmm6
subps xmm7, xmm2 ; xmm7 = data1 - data6 = tmp6
subps xmm6, xmm3 ; xmm6 = data0 - data7 = tmp7
addps xmm0, xmm2 ; xmm0 = data1 + data6 = tmp1
addps xmm5, xmm3 ; xmm5 = data0 + data7 = tmp0
movaps xmm2, XMMWORD [wk(0)] ; xmm2 = (22 23 32 33)
movaps xmm3, XMMWORD [wk(1)] ; xmm3 = (42 43 52 53)
movaps XMMWORD [wk(0)], xmm7 ; wk(0) = tmp6
movaps XMMWORD [wk(1)], xmm6 ; wk(1) = tmp7
movaps xmm7, xmm4 ; transpose coefficients(phase 2)
unpcklps2 xmm4, xmm2 ; xmm4 = (20 21 22 23) = data2
unpckhps2 xmm7, xmm2 ; xmm7 = (30 31 32 33) = data3
movaps xmm6, xmm1 ; transpose coefficients(phase 2)
unpcklps2 xmm1, xmm3 ; xmm1 = (40 41 42 43) = data4
unpckhps2 xmm6, xmm3 ; xmm6 = (50 51 52 53) = data5
movaps xmm2, xmm7
movaps xmm3, xmm4
addps xmm7, xmm1 ; xmm7 = data3 + data4 = tmp3
addps xmm4, xmm6 ; xmm4 = data2 + data5 = tmp2
subps xmm2, xmm1 ; xmm2 = data3 - data4 = tmp4
subps xmm3, xmm6 ; xmm3 = data2 - data5 = tmp5
; -- Even part
movaps xmm1, xmm5
movaps xmm6, xmm0
subps xmm5, xmm7 ; xmm5 = tmp13
subps xmm0, xmm4 ; xmm0 = tmp12
addps xmm1, xmm7 ; xmm1 = tmp10
addps xmm6, xmm4 ; xmm6 = tmp11
addps xmm0, xmm5
mulps xmm0, [rel PD_0_707] ; xmm0 = z1
movaps xmm7, xmm1
movaps xmm4, xmm5
subps xmm1, xmm6 ; xmm1 = data4
subps xmm5, xmm0 ; xmm5 = data6
addps xmm7, xmm6 ; xmm7 = data0
addps xmm4, xmm0 ; xmm4 = data2
movaps XMMWORD [XMMBLOCK(4, 0, rdx, SIZEOF_FAST_FLOAT)], xmm1
movaps XMMWORD [XMMBLOCK(6, 0, rdx, SIZEOF_FAST_FLOAT)], xmm5
movaps XMMWORD [XMMBLOCK(0, 0, rdx, SIZEOF_FAST_FLOAT)], xmm7
movaps XMMWORD [XMMBLOCK(2, 0, rdx, SIZEOF_FAST_FLOAT)], xmm4
; -- Odd part
movaps xmm6, XMMWORD [wk(0)] ; xmm6 = tmp6
movaps xmm0, XMMWORD [wk(1)] ; xmm0 = tmp7
addps xmm2, xmm3 ; xmm2 = tmp10
addps xmm3, xmm6 ; xmm3 = tmp11
addps xmm6, xmm0 ; xmm6 = tmp12, xmm0 = tmp7
mulps xmm3, [rel PD_0_707] ; xmm3 = z3
movaps xmm1, xmm2 ; xmm1 = tmp10
subps xmm2, xmm6
mulps xmm2, [rel PD_0_382] ; xmm2 = z5
mulps xmm1, [rel PD_0_541] ; xmm1 = MULTIPLY(tmp10, FIX_0_541196)
mulps xmm6, [rel PD_1_306] ; xmm6 = MULTIPLY(tmp12, FIX_1_306562)
addps xmm1, xmm2 ; xmm1 = z2
addps xmm6, xmm2 ; xmm6 = z4
movaps xmm5, xmm0
subps xmm0, xmm3 ; xmm0 = z13
addps xmm5, xmm3 ; xmm5 = z11
movaps xmm7, xmm0
movaps xmm4, xmm5
subps xmm0, xmm1 ; xmm0 = data3
subps xmm5, xmm6 ; xmm5 = data7
addps xmm7, xmm1 ; xmm7 = data5
addps xmm4, xmm6 ; xmm4 = data1
movaps XMMWORD [XMMBLOCK(3, 0, rdx, SIZEOF_FAST_FLOAT)], xmm0
movaps XMMWORD [XMMBLOCK(7, 0, rdx, SIZEOF_FAST_FLOAT)], xmm5
movaps XMMWORD [XMMBLOCK(5, 0, rdx, SIZEOF_FAST_FLOAT)], xmm7
movaps XMMWORD [XMMBLOCK(1, 0, rdx, SIZEOF_FAST_FLOAT)], xmm4
add rdx, byte 4 * SIZEOF_FAST_FLOAT
dec rcx
jnz near .columnloop
UNCOLLECT_ARGS 1
lea rsp, [rbp - 8]
pop r15
pop rbp
ret
; For some reason, the OS X linker does not honor the request to align the
; segment unless we do this.
align 32
lib/libjpeg-turbo/simd/x86_64/jfdctint-sse2.asm
Executable
+623
View File
@@ -0,0 +1,623 @@
;
; Accurate integer FDCT (64-bit SSE2)
;
; Copyright 2009 Pierre Ossman <[email protected]> for Cendio AB
; Copyright (C) 2009, 2016, 2020, 2024-2025, D. R. Commander.
; Copyright (C) 2023, Aliaksiej Kandracienka.
;
; Based on the x86 SIMD extension for IJG JPEG library
; Copyright (C) 1999-2006, MIYASAKA Masaru.
; For conditions of distribution and use, see copyright notice in jsimdext.inc
;
; This file should be assembled with NASM (Netwide Assembler) or Yasm.
;
; This file contains a slower but more accurate integer implementation of the
; forward DCT (Discrete Cosine Transform). The following code is based
; directly on the IJG's original jfdctint.c; see jfdctint.c for more details.
%include "jsimdext.inc"
%include "jdct.inc"
; --------------------------------------------------------------------------
%define CONST_BITS 13
%define PASS1_BITS 2
%define DESCALE_P1 (CONST_BITS - PASS1_BITS)
%define DESCALE_P2 (CONST_BITS + PASS1_BITS)
%if CONST_BITS == 13
F_0_298 equ 2446 ; FIX(0.298631336)
F_0_390 equ 3196 ; FIX(0.390180644)
F_0_541 equ 4433 ; FIX(0.541196100)
F_0_765 equ 6270 ; FIX(0.765366865)
F_0_899 equ 7373 ; FIX(0.899976223)
F_1_175 equ 9633 ; FIX(1.175875602)
F_1_501 equ 12299 ; FIX(1.501321110)
F_1_847 equ 15137 ; FIX(1.847759065)
F_1_961 equ 16069 ; FIX(1.961570560)
F_2_053 equ 16819 ; FIX(2.053119869)
F_2_562 equ 20995 ; FIX(2.562915447)
F_3_072 equ 25172 ; FIX(3.072711026)
%else
; NASM cannot do compile-time arithmetic on floating-point constants.
%define DESCALE(x, n) (((x) + (1 << ((n) - 1))) >> (n))
F_0_298 equ DESCALE( 320652955, 30 - CONST_BITS) ; FIX(0.298631336)
F_0_390 equ DESCALE( 418953276, 30 - CONST_BITS) ; FIX(0.390180644)
F_0_541 equ DESCALE( 581104887, 30 - CONST_BITS) ; FIX(0.541196100)
F_0_765 equ DESCALE( 821806413, 30 - CONST_BITS) ; FIX(0.765366865)
F_0_899 equ DESCALE( 966342111, 30 - CONST_BITS) ; FIX(0.899976223)
F_1_175 equ DESCALE(1262586813, 30 - CONST_BITS) ; FIX(1.175875602)
F_1_501 equ DESCALE(1612031267, 30 - CONST_BITS) ; FIX(1.501321110)
F_1_847 equ DESCALE(1984016188, 30 - CONST_BITS) ; FIX(1.847759065)
F_1_961 equ DESCALE(2106220350, 30 - CONST_BITS) ; FIX(1.961570560)
F_2_053 equ DESCALE(2204520673, 30 - CONST_BITS) ; FIX(2.053119869)
F_2_562 equ DESCALE(2751909506, 30 - CONST_BITS) ; FIX(2.562915447)
F_3_072 equ DESCALE(3299298341, 30 - CONST_BITS) ; FIX(3.072711026)
%endif
; --------------------------------------------------------------------------
SECTION SEG_CONST
ALIGNZ 32
GLOBAL_DATA(jconst_fdct_islow_sse2)
EXTN(jconst_fdct_islow_sse2):
PW_F130_F054 times 4 dw (F_0_541 + F_0_765), F_0_541
PW_F054_MF130 times 4 dw F_0_541, (F_0_541 - F_1_847)
PW_MF078_F117 times 4 dw (F_1_175 - F_1_961), F_1_175
PW_F117_F078 times 4 dw F_1_175, (F_1_175 - F_0_390)
PW_MF060_MF089 times 4 dw (F_0_298 - F_0_899), -F_0_899
PW_MF089_F060 times 4 dw -F_0_899, (F_1_501 - F_0_899)
PW_MF050_MF256 times 4 dw (F_2_053 - F_2_562), -F_2_562
PW_MF256_F050 times 4 dw -F_2_562, (F_3_072 - F_2_562)
PD_DESCALE_P1 times 4 dd 1 << (DESCALE_P1 - 1)
PD_DESCALE_P2 times 4 dd 1 << (DESCALE_P2 - 1)
PW_DESCALE_P2X times 8 dw 1 << (PASS1_BITS - 1)
ALIGNZ 32
; --------------------------------------------------------------------------
SECTION SEG_TEXT
BITS 64
; Perform the forward DCT on one block of samples.
;
; GLOBAL(void)
; jsimd_fdct_islow_sse2(DCTELEM *data)
;
; r10 = DCTELEM *data
%define wk(i) r15 - (WK_NUM - (i)) * SIZEOF_XMMWORD ; xmmword wk[WK_NUM]
%define WK_NUM 6
align 32
GLOBAL_FUNCTION(jsimd_fdct_islow_sse2)
EXTN(jsimd_fdct_islow_sse2):
ENDBR64
push rbp
mov rbp, rsp
push r15
and rsp, byte (-SIZEOF_XMMWORD) ; align to 128 bits
; Allocate stack space for wk array. r15 is used to access it.
mov r15, rsp
sub rsp, byte (SIZEOF_XMMWORD * WK_NUM)
COLLECT_ARGS 1
; ---- Pass 1: process rows.
mov rdx, r10 ; (DCTELEM *)
movdqa xmm0, XMMWORD [XMMBLOCK(0, 0, rdx, SIZEOF_DCTELEM)]
movdqa xmm1, XMMWORD [XMMBLOCK(1, 0, rdx, SIZEOF_DCTELEM)]
movdqa xmm2, XMMWORD [XMMBLOCK(2, 0, rdx, SIZEOF_DCTELEM)]
movdqa xmm3, XMMWORD [XMMBLOCK(3, 0, rdx, SIZEOF_DCTELEM)]
; xmm0 = (00 01 02 03 04 05 06 07)
; xmm2 = (20 21 22 23 24 25 26 27)
; xmm1 = (10 11 12 13 14 15 16 17)
; xmm3 = (30 31 32 33 34 35 36 37)
movdqa xmm4, xmm0 ; transpose coefficients(phase 1)
punpcklwd xmm0, xmm1 ; xmm0 = (00 10 01 11 02 12 03 13)
punpckhwd xmm4, xmm1 ; xmm4 = (04 14 05 15 06 16 07 17)
movdqa xmm5, xmm2 ; transpose coefficients(phase 1)
punpcklwd xmm2, xmm3 ; xmm2 = (20 30 21 31 22 32 23 33)
punpckhwd xmm5, xmm3 ; xmm5 = (24 34 25 35 26 36 27 37)
movdqa xmm6, XMMWORD [XMMBLOCK(4, 0, rdx, SIZEOF_DCTELEM)]
movdqa xmm7, XMMWORD [XMMBLOCK(5, 0, rdx, SIZEOF_DCTELEM)]
movdqa xmm1, XMMWORD [XMMBLOCK(6, 0, rdx, SIZEOF_DCTELEM)]
movdqa xmm3, XMMWORD [XMMBLOCK(7, 0, rdx, SIZEOF_DCTELEM)]
; xmm6 = ( 4 12 20 28 36 44 52 60)
; xmm1 = ( 6 14 22 30 38 46 54 62)
; xmm7 = ( 5 13 21 29 37 45 53 61)
; xmm3 = ( 7 15 23 31 39 47 55 63)
movdqa XMMWORD [wk(0)], xmm2 ; wk(0) = (20 30 21 31 22 32 23 33)
movdqa XMMWORD [wk(1)], xmm5 ; wk(1) = (24 34 25 35 26 36 27 37)
movdqa xmm2, xmm6 ; transpose coefficients(phase 1)
punpcklwd xmm6, xmm7 ; xmm6 = (40 50 41 51 42 52 43 53)
punpckhwd xmm2, xmm7 ; xmm2 = (44 54 45 55 46 56 47 57)
movdqa xmm5, xmm1 ; transpose coefficients(phase 1)
punpcklwd xmm1, xmm3 ; xmm1 = (60 70 61 71 62 72 63 73)
punpckhwd xmm5, xmm3 ; xmm5 = (64 74 65 75 66 76 67 77)
movdqa xmm7, xmm6 ; transpose coefficients(phase 2)
punpckldq xmm6, xmm1 ; xmm6 = (40 50 60 70 41 51 61 71)
punpckhdq xmm7, xmm1 ; xmm7 = (42 52 62 72 43 53 63 73)
movdqa xmm3, xmm2 ; transpose coefficients(phase 2)
punpckldq xmm2, xmm5 ; xmm2 = (44 54 64 74 45 55 65 75)
punpckhdq xmm3, xmm5 ; xmm3 = (46 56 66 76 47 57 67 77)
movdqa xmm1, XMMWORD [wk(0)] ; xmm1 = (20 30 21 31 22 32 23 33)
movdqa xmm5, XMMWORD [wk(1)] ; xmm5 = (24 34 25 35 26 36 27 37)
movdqa XMMWORD [wk(2)], xmm7 ; wk(2) = (42 52 62 72 43 53 63 73)
movdqa XMMWORD [wk(3)], xmm2 ; wk(3) = (44 54 64 74 45 55 65 75)
movdqa xmm7, xmm0 ; transpose coefficients(phase 2)
punpckldq xmm0, xmm1 ; xmm0 = (00 10 20 30 01 11 21 31)
punpckhdq xmm7, xmm1 ; xmm7 = (02 12 22 32 03 13 23 33)
movdqa xmm2, xmm4 ; transpose coefficients(phase 2)
punpckldq xmm4, xmm5 ; xmm4 = (04 14 24 34 05 15 25 35)
punpckhdq xmm2, xmm5 ; xmm2 = (06 16 26 36 07 17 27 37)
movdqa xmm1, xmm0 ; transpose coefficients(phase 3)
punpcklqdq xmm0, xmm6 ; xmm0 = (00 10 20 30 40 50 60 70) = data0
punpckhqdq xmm1, xmm6 ; xmm1 = (01 11 21 31 41 51 61 71) = data1
movdqa xmm5, xmm2 ; transpose coefficients(phase 3)
punpcklqdq xmm2, xmm3 ; xmm2 = (06 16 26 36 46 56 66 76) = data6
punpckhqdq xmm5, xmm3 ; xmm5 = (07 17 27 37 47 57 67 77) = data7
movdqa xmm6, xmm1
movdqa xmm3, xmm0
psubw xmm1, xmm2 ; xmm1 = data1 - data6 = tmp6
psubw xmm0, xmm5 ; xmm0 = data0 - data7 = tmp7
paddw xmm6, xmm2 ; xmm6 = data1 + data6 = tmp1
paddw xmm3, xmm5 ; xmm3 = data0 + data7 = tmp0
movdqa xmm2, XMMWORD [wk(2)] ; xmm2 = (42 52 62 72 43 53 63 73)
movdqa xmm5, XMMWORD [wk(3)] ; xmm5 = (44 54 64 74 45 55 65 75)
movdqa XMMWORD [wk(0)], xmm1 ; wk(0) = tmp6
movdqa XMMWORD [wk(1)], xmm0 ; wk(1) = tmp7
movdqa xmm1, xmm7 ; transpose coefficients(phase 3)
punpcklqdq xmm7, xmm2 ; xmm7 = (02 12 22 32 42 52 62 72) = data2
punpckhqdq xmm1, xmm2 ; xmm1 = (03 13 23 33 43 53 63 73) = data3
movdqa xmm0, xmm4 ; transpose coefficients(phase 3)
punpcklqdq xmm4, xmm5 ; xmm4 = (04 14 24 34 44 54 64 74) = data4
punpckhqdq xmm0, xmm5 ; xmm0 = (05 15 25 35 45 55 65 75) = data5
movdqa xmm2, xmm1
movdqa xmm5, xmm7
paddw xmm1, xmm4 ; xmm1 = data3 + data4 = tmp3
paddw xmm7, xmm0 ; xmm7 = data2 + data5 = tmp2
psubw xmm2, xmm4 ; xmm2 = data3 - data4 = tmp4
psubw xmm5, xmm0 ; xmm5 = data2 - data5 = tmp5
; -- Even part
movdqa xmm4, xmm3
movdqa xmm0, xmm6
paddw xmm3, xmm1 ; xmm3 = tmp10
paddw xmm6, xmm7 ; xmm6 = tmp11
psubw xmm4, xmm1 ; xmm4 = tmp13
psubw xmm0, xmm7 ; xmm0 = tmp12
movdqa xmm1, xmm3
paddw xmm3, xmm6 ; xmm3 = tmp10 + tmp11
psubw xmm1, xmm6 ; xmm1 = tmp10 - tmp11
psllw xmm3, PASS1_BITS ; xmm3 = data0
psllw xmm1, PASS1_BITS ; xmm1 = data4
movdqa XMMWORD [wk(2)], xmm3 ; wk(2) = data0
movdqa XMMWORD [wk(3)], xmm1 ; wk(3) = data4
; (Original)
; z1 = (tmp12 + tmp13) * 0.541196100;
; data2 = z1 + tmp13 * 0.765366865;
; data6 = z1 + tmp12 * -1.847759065;
;
; (This implementation)
; data2 = tmp13 * (0.541196100 + 0.765366865) + tmp12 * 0.541196100;
; data6 = tmp13 * 0.541196100 + tmp12 * (0.541196100 - 1.847759065);
movdqa xmm7, xmm4 ; xmm4 = tmp13
movdqa xmm6, xmm4
punpcklwd xmm7, xmm0 ; xmm0 = tmp12
punpckhwd xmm6, xmm0
movdqa xmm4, xmm7
movdqa xmm0, xmm6
pmaddwd xmm7, [rel PW_F130_F054] ; xmm7 = data2L
pmaddwd xmm6, [rel PW_F130_F054] ; xmm6 = data2H
pmaddwd xmm4, [rel PW_F054_MF130] ; xmm4 = data6L
pmaddwd xmm0, [rel PW_F054_MF130] ; xmm0 = data6H
paddd xmm7, [rel PD_DESCALE_P1]
paddd xmm6, [rel PD_DESCALE_P1]
psrad xmm7, DESCALE_P1
psrad xmm6, DESCALE_P1
paddd xmm4, [rel PD_DESCALE_P1]
paddd xmm0, [rel PD_DESCALE_P1]
psrad xmm4, DESCALE_P1
psrad xmm0, DESCALE_P1
packssdw xmm7, xmm6 ; xmm7 = data2
packssdw xmm4, xmm0 ; xmm4 = data6
movdqa XMMWORD [wk(4)], xmm7 ; wk(4) = data2
movdqa XMMWORD [wk(5)], xmm4 ; wk(5) = data6
; -- Odd part
movdqa xmm3, XMMWORD [wk(0)] ; xmm3 = tmp6
movdqa xmm1, XMMWORD [wk(1)] ; xmm1 = tmp7
movdqa xmm6, xmm2 ; xmm2 = tmp4
movdqa xmm0, xmm5 ; xmm5 = tmp5
paddw xmm6, xmm3 ; xmm6 = z3
paddw xmm0, xmm1 ; xmm0 = z4
; (Original)
; z5 = (z3 + z4) * 1.175875602;
; z3 = z3 * -1.961570560; z4 = z4 * -0.390180644;
; z3 += z5; z4 += z5;
;
; (This implementation)
; z3 = z3 * (1.175875602 - 1.961570560) + z4 * 1.175875602;
; z4 = z3 * 1.175875602 + z4 * (1.175875602 - 0.390180644);
movdqa xmm7, xmm6
movdqa xmm4, xmm6
punpcklwd xmm7, xmm0
punpckhwd xmm4, xmm0
movdqa xmm6, xmm7
movdqa xmm0, xmm4
pmaddwd xmm7, [rel PW_MF078_F117] ; xmm7 = z3L
pmaddwd xmm4, [rel PW_MF078_F117] ; xmm4 = z3H
pmaddwd xmm6, [rel PW_F117_F078] ; xmm6 = z4L
pmaddwd xmm0, [rel PW_F117_F078] ; xmm0 = z4H
movdqa XMMWORD [wk(0)], xmm7 ; wk(0) = z3L
movdqa XMMWORD [wk(1)], xmm4 ; wk(1) = z3H
; (Original)
; z1 = tmp4 + tmp7; z2 = tmp5 + tmp6;
; tmp4 = tmp4 * 0.298631336; tmp5 = tmp5 * 2.053119869;
; tmp6 = tmp6 * 3.072711026; tmp7 = tmp7 * 1.501321110;
; z1 = z1 * -0.899976223; z2 = z2 * -2.562915447;
; data7 = tmp4 + z1 + z3; data5 = tmp5 + z2 + z4;
; data3 = tmp6 + z2 + z3; data1 = tmp7 + z1 + z4;
;
; (This implementation)
; tmp4 = tmp4 * (0.298631336 - 0.899976223) + tmp7 * -0.899976223;
; tmp5 = tmp5 * (2.053119869 - 2.562915447) + tmp6 * -2.562915447;
; tmp6 = tmp5 * -2.562915447 + tmp6 * (3.072711026 - 2.562915447);
; tmp7 = tmp4 * -0.899976223 + tmp7 * (1.501321110 - 0.899976223);
; data7 = tmp4 + z3; data5 = tmp5 + z4;
; data3 = tmp6 + z3; data1 = tmp7 + z4;
movdqa xmm7, xmm2
movdqa xmm4, xmm2
punpcklwd xmm7, xmm1
punpckhwd xmm4, xmm1
movdqa xmm2, xmm7
movdqa xmm1, xmm4
pmaddwd xmm7, [rel PW_MF060_MF089] ; xmm7 = tmp4L
pmaddwd xmm4, [rel PW_MF060_MF089] ; xmm4 = tmp4H
pmaddwd xmm2, [rel PW_MF089_F060] ; xmm2 = tmp7L
pmaddwd xmm1, [rel PW_MF089_F060] ; xmm1 = tmp7H
paddd xmm7, XMMWORD [wk(0)] ; xmm7 = data7L
paddd xmm4, XMMWORD [wk(1)] ; xmm4 = data7H
paddd xmm2, xmm6 ; xmm2 = data1L
paddd xmm1, xmm0 ; xmm1 = data1H
paddd xmm7, [rel PD_DESCALE_P1]
paddd xmm4, [rel PD_DESCALE_P1]
psrad xmm7, DESCALE_P1
psrad xmm4, DESCALE_P1
paddd xmm2, [rel PD_DESCALE_P1]
paddd xmm1, [rel PD_DESCALE_P1]
psrad xmm2, DESCALE_P1
psrad xmm1, DESCALE_P1
packssdw xmm7, xmm4 ; xmm7 = data7
packssdw xmm2, xmm1 ; xmm2 = data1
movdqa xmm4, xmm5
movdqa xmm1, xmm5
punpcklwd xmm4, xmm3
punpckhwd xmm1, xmm3
movdqa xmm5, xmm4
movdqa xmm3, xmm1
pmaddwd xmm4, [rel PW_MF050_MF256] ; xmm4 = tmp5L
pmaddwd xmm1, [rel PW_MF050_MF256] ; xmm1 = tmp5H
pmaddwd xmm5, [rel PW_MF256_F050] ; xmm5 = tmp6L
pmaddwd xmm3, [rel PW_MF256_F050] ; xmm3 = tmp6H
paddd xmm4, xmm6 ; xmm4 = data5L
paddd xmm1, xmm0 ; xmm1 = data5H
paddd xmm5, XMMWORD [wk(0)] ; xmm5 = data3L
paddd xmm3, XMMWORD [wk(1)] ; xmm3 = data3H
paddd xmm4, [rel PD_DESCALE_P1]
paddd xmm1, [rel PD_DESCALE_P1]
psrad xmm4, DESCALE_P1
psrad xmm1, DESCALE_P1
paddd xmm5, [rel PD_DESCALE_P1]
paddd xmm3, [rel PD_DESCALE_P1]
psrad xmm5, DESCALE_P1
psrad xmm3, DESCALE_P1
packssdw xmm4, xmm1 ; xmm4 = data5
packssdw xmm5, xmm3 ; xmm5 = data3
; ---- Pass 2: process columns.
movdqa xmm6, XMMWORD [wk(2)] ; xmm6 = col0
movdqa xmm0, XMMWORD [wk(4)] ; xmm0 = col2
; xmm6 = (00 10 20 30 40 50 60 70)
; xmm0 = (02 12 22 32 42 52 62 72)
; xmm2 = (01 11 21 31 41 51 61 71)
; xmm5 = (03 13 23 33 43 53 63 73)
movdqa xmm1, xmm6 ; transpose coefficients(phase 1)
punpcklwd xmm6, xmm2 ; xmm6 = (00 01 10 11 20 21 30 31)
punpckhwd xmm1, xmm2 ; xmm1 = (40 41 50 51 60 61 70 71)
movdqa xmm3, xmm0 ; transpose coefficients(phase 1)
punpcklwd xmm0, xmm5 ; xmm0 = (02 03 12 13 22 23 32 33)
punpckhwd xmm3, xmm5 ; xmm3 = (42 43 52 53 62 63 72 73)
movdqa xmm2, XMMWORD [wk(3)] ; xmm2 = col4
movdqa xmm5, XMMWORD [wk(5)] ; xmm5 = col6
; xmm2 = (04 14 24 34 44 54 64 74)
; xmm5 = (06 16 26 36 46 56 66 76)
; xmm4 = (05 15 25 35 45 55 65 75)
; xmm7 = (07 17 27 37 47 57 67 77)
movdqa XMMWORD [wk(0)], xmm0 ; wk(0) = (02 03 12 13 22 23 32 33)
movdqa XMMWORD [wk(1)], xmm3 ; wk(1) = (42 43 52 53 62 63 72 73)
movdqa xmm0, xmm2 ; transpose coefficients(phase 1)
punpcklwd xmm2, xmm4 ; xmm2 = (04 05 14 15 24 25 34 35)
punpckhwd xmm0, xmm4 ; xmm0 = (44 45 54 55 64 65 74 75)
movdqa xmm3, xmm5 ; transpose coefficients(phase 1)
punpcklwd xmm5, xmm7 ; xmm5 = (06 07 16 17 26 27 36 37)
punpckhwd xmm3, xmm7 ; xmm3 = (46 47 56 57 66 67 76 77)
movdqa xmm4, xmm2 ; transpose coefficients(phase 2)
punpckldq xmm2, xmm5 ; xmm2 = (04 05 06 07 14 15 16 17)
punpckhdq xmm4, xmm5 ; xmm4 = (24 25 26 27 34 35 36 37)
movdqa xmm7, xmm0 ; transpose coefficients(phase 2)
punpckldq xmm0, xmm3 ; xmm0 = (44 45 46 47 54 55 56 57)
punpckhdq xmm7, xmm3 ; xmm7 = (64 65 66 67 74 75 76 77)
movdqa xmm5, XMMWORD [wk(0)] ; xmm5 = (02 03 12 13 22 23 32 33)
movdqa xmm3, XMMWORD [wk(1)] ; xmm3 = (42 43 52 53 62 63 72 73)
movdqa XMMWORD [wk(2)], xmm4 ; wk(2) = (24 25 26 27 34 35 36 37)
movdqa XMMWORD [wk(3)], xmm0 ; wk(3) = (44 45 46 47 54 55 56 57)
movdqa xmm4, xmm6 ; transpose coefficients(phase 2)
punpckldq xmm6, xmm5 ; xmm6 = (00 01 02 03 10 11 12 13)
punpckhdq xmm4, xmm5 ; xmm4 = (20 21 22 23 30 31 32 33)
movdqa xmm0, xmm1 ; transpose coefficients(phase 2)
punpckldq xmm1, xmm3 ; xmm1 = (40 41 42 43 50 51 52 53)
punpckhdq xmm0, xmm3 ; xmm0 = (60 61 62 63 70 71 72 73)
movdqa xmm5, xmm6 ; transpose coefficients(phase 3)
punpcklqdq xmm6, xmm2 ; xmm6 = (00 01 02 03 04 05 06 07) = data0
punpckhqdq xmm5, xmm2 ; xmm5 = (10 11 12 13 14 15 16 17) = data1
movdqa xmm3, xmm0 ; transpose coefficients(phase 3)
punpcklqdq xmm0, xmm7 ; xmm0 = (60 61 62 63 64 65 66 67) = data6
punpckhqdq xmm3, xmm7 ; xmm3 = (70 71 72 73 74 75 76 77) = data7
movdqa xmm2, xmm5
movdqa xmm7, xmm6
psubw xmm5, xmm0 ; xmm5 = data1 - data6 = tmp6
psubw xmm6, xmm3 ; xmm6 = data0 - data7 = tmp7
paddw xmm2, xmm0 ; xmm2 = data1 + data6 = tmp1
paddw xmm7, xmm3 ; xmm7 = data0 + data7 = tmp0
movdqa xmm0, XMMWORD [wk(2)] ; xmm0 = (24 25 26 27 34 35 36 37)
movdqa xmm3, XMMWORD [wk(3)] ; xmm3 = (44 45 46 47 54 55 56 57)
movdqa XMMWORD [wk(0)], xmm5 ; wk(0) = tmp6
movdqa XMMWORD [wk(1)], xmm6 ; wk(1) = tmp7
movdqa xmm5, xmm4 ; transpose coefficients(phase 3)
punpcklqdq xmm4, xmm0 ; xmm4 = (20 21 22 23 24 25 26 27) = data2
punpckhqdq xmm5, xmm0 ; xmm5 = (30 31 32 33 34 35 36 37) = data3
movdqa xmm6, xmm1 ; transpose coefficients(phase 3)
punpcklqdq xmm1, xmm3 ; xmm1 = (40 41 42 43 44 45 46 47) = data4
punpckhqdq xmm6, xmm3 ; xmm6 = (50 51 52 53 54 55 56 57) = data5
movdqa xmm0, xmm5
movdqa xmm3, xmm4
paddw xmm5, xmm1 ; xmm5 = data3 + data4 = tmp3
paddw xmm4, xmm6 ; xmm4 = data2 + data5 = tmp2
psubw xmm0, xmm1 ; xmm0 = data3 - data4 = tmp4
psubw xmm3, xmm6 ; xmm3 = data2 - data5 = tmp5
; -- Even part
movdqa xmm1, xmm7
movdqa xmm6, xmm2
paddw xmm7, xmm5 ; xmm7 = tmp10
paddw xmm2, xmm4 ; xmm2 = tmp11
psubw xmm1, xmm5 ; xmm1 = tmp13
psubw xmm6, xmm4 ; xmm6 = tmp12
movdqa xmm5, xmm7
paddw xmm7, xmm2 ; xmm7 = tmp10 + tmp11
psubw xmm5, xmm2 ; xmm5 = tmp10 - tmp11
paddw xmm7, [rel PW_DESCALE_P2X]
paddw xmm5, [rel PW_DESCALE_P2X]
psraw xmm7, PASS1_BITS ; xmm7 = data0
psraw xmm5, PASS1_BITS ; xmm5 = data4
movdqa XMMWORD [XMMBLOCK(0, 0, rdx, SIZEOF_DCTELEM)], xmm7
movdqa XMMWORD [XMMBLOCK(4, 0, rdx, SIZEOF_DCTELEM)], xmm5
; (Original)
; z1 = (tmp12 + tmp13) * 0.541196100;
; data2 = z1 + tmp13 * 0.765366865;
; data6 = z1 + tmp12 * -1.847759065;
;
; (This implementation)
; data2 = tmp13 * (0.541196100 + 0.765366865) + tmp12 * 0.541196100;
; data6 = tmp13 * 0.541196100 + tmp12 * (0.541196100 - 1.847759065);
movdqa xmm4, xmm1 ; xmm1 = tmp13
movdqa xmm2, xmm1
punpcklwd xmm4, xmm6 ; xmm6 = tmp12
punpckhwd xmm2, xmm6
movdqa xmm1, xmm4
movdqa xmm6, xmm2
pmaddwd xmm4, [rel PW_F130_F054] ; xmm4 = data2L
pmaddwd xmm2, [rel PW_F130_F054] ; xmm2 = data2H
pmaddwd xmm1, [rel PW_F054_MF130] ; xmm1 = data6L
pmaddwd xmm6, [rel PW_F054_MF130] ; xmm6 = data6H
paddd xmm4, [rel PD_DESCALE_P2]
paddd xmm2, [rel PD_DESCALE_P2]
psrad xmm4, DESCALE_P2
psrad xmm2, DESCALE_P2
paddd xmm1, [rel PD_DESCALE_P2]
paddd xmm6, [rel PD_DESCALE_P2]
psrad xmm1, DESCALE_P2
psrad xmm6, DESCALE_P2
packssdw xmm4, xmm2 ; xmm4 = data2
packssdw xmm1, xmm6 ; xmm1 = data6
movdqa XMMWORD [XMMBLOCK(2, 0, rdx, SIZEOF_DCTELEM)], xmm4
movdqa XMMWORD [XMMBLOCK(6, 0, rdx, SIZEOF_DCTELEM)], xmm1
; -- Odd part
movdqa xmm7, XMMWORD [wk(0)] ; xmm7 = tmp6
movdqa xmm5, XMMWORD [wk(1)] ; xmm5 = tmp7
movdqa xmm2, xmm0 ; xmm0 = tmp4
movdqa xmm6, xmm3 ; xmm3 = tmp5
paddw xmm2, xmm7 ; xmm2 = z3
paddw xmm6, xmm5 ; xmm6 = z4
; (Original)
; z5 = (z3 + z4) * 1.175875602;
; z3 = z3 * -1.961570560; z4 = z4 * -0.390180644;
; z3 += z5; z4 += z5;
;
; (This implementation)
; z3 = z3 * (1.175875602 - 1.961570560) + z4 * 1.175875602;
; z4 = z3 * 1.175875602 + z4 * (1.175875602 - 0.390180644);
movdqa xmm4, xmm2
movdqa xmm1, xmm2
punpcklwd xmm4, xmm6
punpckhwd xmm1, xmm6
movdqa xmm2, xmm4
movdqa xmm6, xmm1
pmaddwd xmm4, [rel PW_MF078_F117] ; xmm4 = z3L
pmaddwd xmm1, [rel PW_MF078_F117] ; xmm1 = z3H
pmaddwd xmm2, [rel PW_F117_F078] ; xmm2 = z4L
pmaddwd xmm6, [rel PW_F117_F078] ; xmm6 = z4H
movdqa XMMWORD [wk(0)], xmm4 ; wk(0) = z3L
movdqa XMMWORD [wk(1)], xmm1 ; wk(1) = z3H
; (Original)
; z1 = tmp4 + tmp7; z2 = tmp5 + tmp6;
; tmp4 = tmp4 * 0.298631336; tmp5 = tmp5 * 2.053119869;
; tmp6 = tmp6 * 3.072711026; tmp7 = tmp7 * 1.501321110;
; z1 = z1 * -0.899976223; z2 = z2 * -2.562915447;
; data7 = tmp4 + z1 + z3; data5 = tmp5 + z2 + z4;
; data3 = tmp6 + z2 + z3; data1 = tmp7 + z1 + z4;
;
; (This implementation)
; tmp4 = tmp4 * (0.298631336 - 0.899976223) + tmp7 * -0.899976223;
; tmp5 = tmp5 * (2.053119869 - 2.562915447) + tmp6 * -2.562915447;
; tmp6 = tmp5 * -2.562915447 + tmp6 * (3.072711026 - 2.562915447);
; tmp7 = tmp4 * -0.899976223 + tmp7 * (1.501321110 - 0.899976223);
; data7 = tmp4 + z3; data5 = tmp5 + z4;
; data3 = tmp6 + z3; data1 = tmp7 + z4;
movdqa xmm4, xmm0
movdqa xmm1, xmm0
punpcklwd xmm4, xmm5
punpckhwd xmm1, xmm5
movdqa xmm0, xmm4
movdqa xmm5, xmm1
pmaddwd xmm4, [rel PW_MF060_MF089] ; xmm4 = tmp4L
pmaddwd xmm1, [rel PW_MF060_MF089] ; xmm1 = tmp4H
pmaddwd xmm0, [rel PW_MF089_F060] ; xmm0 = tmp7L
pmaddwd xmm5, [rel PW_MF089_F060] ; xmm5 = tmp7H
paddd xmm4, XMMWORD [wk(0)] ; xmm4 = data7L
paddd xmm1, XMMWORD [wk(1)] ; xmm1 = data7H
paddd xmm0, xmm2 ; xmm0 = data1L
paddd xmm5, xmm6 ; xmm5 = data1H
paddd xmm4, [rel PD_DESCALE_P2]
paddd xmm1, [rel PD_DESCALE_P2]
psrad xmm4, DESCALE_P2
psrad xmm1, DESCALE_P2
paddd xmm0, [rel PD_DESCALE_P2]
paddd xmm5, [rel PD_DESCALE_P2]
psrad xmm0, DESCALE_P2
psrad xmm5, DESCALE_P2
packssdw xmm4, xmm1 ; xmm4 = data7
packssdw xmm0, xmm5 ; xmm0 = data1
movdqa XMMWORD [XMMBLOCK(7, 0, rdx, SIZEOF_DCTELEM)], xmm4
movdqa XMMWORD [XMMBLOCK(1, 0, rdx, SIZEOF_DCTELEM)], xmm0
movdqa xmm1, xmm3
movdqa xmm5, xmm3
punpcklwd xmm1, xmm7
punpckhwd xmm5, xmm7
movdqa xmm3, xmm1
movdqa xmm7, xmm5
pmaddwd xmm1, [rel PW_MF050_MF256] ; xmm1 = tmp5L
pmaddwd xmm5, [rel PW_MF050_MF256] ; xmm5 = tmp5H
pmaddwd xmm3, [rel PW_MF256_F050] ; xmm3 = tmp6L
pmaddwd xmm7, [rel PW_MF256_F050] ; xmm7 = tmp6H
paddd xmm1, xmm2 ; xmm1 = data5L
paddd xmm5, xmm6 ; xmm5 = data5H
paddd xmm3, XMMWORD [wk(0)] ; xmm3 = data3L
paddd xmm7, XMMWORD [wk(1)] ; xmm7 = data3H
paddd xmm1, [rel PD_DESCALE_P2]
paddd xmm5, [rel PD_DESCALE_P2]
psrad xmm1, DESCALE_P2
psrad xmm5, DESCALE_P2
paddd xmm3, [rel PD_DESCALE_P2]
paddd xmm7, [rel PD_DESCALE_P2]
psrad xmm3, DESCALE_P2
psrad xmm7, DESCALE_P2
packssdw xmm1, xmm5 ; xmm1 = data5
packssdw xmm3, xmm7 ; xmm3 = data3
movdqa XMMWORD [XMMBLOCK(5, 0, rdx, SIZEOF_DCTELEM)], xmm1
movdqa XMMWORD [XMMBLOCK(3, 0, rdx, SIZEOF_DCTELEM)], xmm3
UNCOLLECT_ARGS 1
lea rsp, [rbp - 8]
pop r15
pop rbp
ret
; For some reason, the OS X linker does not honor the request to align the
; segment unless we do this.
align 32
lib/libjpeg-turbo/simd/x86_64/jidctfst-sse2.asm
Executable
+508
View File
@@ -0,0 +1,508 @@
;
; Fast integer IDCT (64-bit SSE2)
;
; Copyright 2009 Pierre Ossman <[email protected]> for Cendio AB
; Copyright (C) 2009, 2016, 2024-2025, D. R. Commander.
; Copyright (C) 2018, Matthias Räncker.
; Copyright (C) 2023, Aliaksiej Kandracienka.
;
; Based on the x86 SIMD extension for IJG JPEG library
; Copyright (C) 1999-2006, MIYASAKA Masaru.
; For conditions of distribution and use, see copyright notice in jsimdext.inc
;
; This file should be assembled with NASM (Netwide Assembler) or Yasm.
;
; This file contains a fast, not so accurate integer implementation of the
; inverse DCT (Discrete Cosine Transform). The following code is based
; directly on the IJG's original jidctfst.c; see jidctfst.c for more details.
%include "jsimdext.inc"
%include "jdct.inc"
; --------------------------------------------------------------------------
%define CONST_BITS 8 ; 14 is also OK.
%define PASS1_BITS 2
%if IFAST_SCALE_BITS != PASS1_BITS
%error "'IFAST_SCALE_BITS' must be equal to 'PASS1_BITS'."
%endif
%if CONST_BITS == 8
F_1_082 equ 277 ; FIX(1.082392200)
F_1_414 equ 362 ; FIX(1.414213562)
F_1_847 equ 473 ; FIX(1.847759065)
F_2_613 equ 669 ; FIX(2.613125930)
F_1_613 equ (F_2_613 - 256) ; FIX(2.613125930) - FIX(1)
%else
; NASM cannot do compile-time arithmetic on floating-point constants.
%define DESCALE(x, n) (((x) + (1 << ((n) - 1))) >> (n))
F_1_082 equ DESCALE(1162209775, 30 - CONST_BITS) ; FIX(1.082392200)
F_1_414 equ DESCALE(1518500249, 30 - CONST_BITS) ; FIX(1.414213562)
F_1_847 equ DESCALE(1984016188, 30 - CONST_BITS) ; FIX(1.847759065)
F_2_613 equ DESCALE(2805822602, 30 - CONST_BITS) ; FIX(2.613125930)
F_1_613 equ (F_2_613 - (1 << CONST_BITS)) ; FIX(2.613125930) - FIX(1)
%endif
; --------------------------------------------------------------------------
SECTION SEG_CONST
; PRE_MULTIPLY_SCALE_BITS <= 2 (to avoid overflow)
; CONST_BITS + CONST_SHIFT + PRE_MULTIPLY_SCALE_BITS == 16 (for pmulhw)
%define PRE_MULTIPLY_SCALE_BITS 2
%define CONST_SHIFT (16 - PRE_MULTIPLY_SCALE_BITS - CONST_BITS)
ALIGNZ 32
GLOBAL_DATA(jconst_idct_ifast_sse2)
EXTN(jconst_idct_ifast_sse2):
PW_F1414 times 8 dw F_1_414 << CONST_SHIFT
PW_F1847 times 8 dw F_1_847 << CONST_SHIFT
PW_MF1613 times 8 dw -F_1_613 << CONST_SHIFT
PW_F1082 times 8 dw F_1_082 << CONST_SHIFT
PB_CENTERJSAMP times 16 db CENTERJSAMPLE
ALIGNZ 32
; --------------------------------------------------------------------------
SECTION SEG_TEXT
BITS 64
; Perform dequantization and inverse DCT on one block of coefficients.
;
; GLOBAL(void)
; jsimd_idct_ifast_sse2(void *dct_table, JCOEFPTR coef_block,
; JSAMPARRAY output_buf, JDIMENSION output_col)
;
; r10 = jpeg_component_info *compptr
; r11 = JCOEFPTR coef_block
; r12 = JSAMPARRAY output_buf
; r13d = JDIMENSION output_col
%define wk(i) r15 - (WK_NUM - (i)) * SIZEOF_XMMWORD
; xmmword wk[WK_NUM]
%define WK_NUM 2
align 32
GLOBAL_FUNCTION(jsimd_idct_ifast_sse2)
EXTN(jsimd_idct_ifast_sse2):
ENDBR64
push rbp
mov rbp, rsp
push r15
and rsp, byte (-SIZEOF_XMMWORD) ; align to 128 bits
; Allocate stack space for wk array. r15 is used to access it.
mov r15, rsp
sub rsp, byte (SIZEOF_XMMWORD * WK_NUM)
COLLECT_ARGS 4
; ---- Pass 1: process columns from input.
mov rdx, r10 ; quantptr
mov rsi, r11 ; inptr
%ifndef NO_ZERO_COLUMN_TEST_IFAST_SSE2
mov eax, dword [DWBLOCK(1, 0, rsi, SIZEOF_JCOEF)]
or eax, dword [DWBLOCK(2, 0, rsi, SIZEOF_JCOEF)]
jnz near .columnDCT
movdqa xmm0, XMMWORD [XMMBLOCK(1, 0, rsi, SIZEOF_JCOEF)]
movdqa xmm1, XMMWORD [XMMBLOCK(2, 0, rsi, SIZEOF_JCOEF)]
por xmm0, XMMWORD [XMMBLOCK(3, 0, rsi, SIZEOF_JCOEF)]
por xmm1, XMMWORD [XMMBLOCK(4, 0, rsi, SIZEOF_JCOEF)]
por xmm0, XMMWORD [XMMBLOCK(5, 0, rsi, SIZEOF_JCOEF)]
por xmm1, XMMWORD [XMMBLOCK(6, 0, rsi, SIZEOF_JCOEF)]
por xmm0, XMMWORD [XMMBLOCK(7, 0, rsi, SIZEOF_JCOEF)]
por xmm1, xmm0
packsswb xmm1, xmm1
packsswb xmm1, xmm1
movd eax, xmm1
test rax, rax
jnz short .columnDCT
; -- AC terms all zero
movdqa xmm0, XMMWORD [XMMBLOCK(0, 0, rsi, SIZEOF_JCOEF)]
pmullw xmm0, XMMWORD [XMMBLOCK(0, 0, rdx, SIZEOF_ISLOW_MULT_TYPE)]
movdqa xmm7, xmm0 ; xmm0 = in0 = (00 01 02 03 04 05 06 07)
punpcklwd xmm0, xmm0 ; xmm0 = (00 00 01 01 02 02 03 03)
punpckhwd xmm7, xmm7 ; xmm7 = (04 04 05 05 06 06 07 07)
pshufd xmm6, xmm0, 0x00 ; xmm6 = col0 = (00 00 00 00 00 00 00 00)
pshufd xmm2, xmm0, 0x55 ; xmm2 = col1 = (01 01 01 01 01 01 01 01)
pshufd xmm5, xmm0, 0xAA ; xmm5 = col2 = (02 02 02 02 02 02 02 02)
pshufd xmm0, xmm0, 0xFF ; xmm0 = col3 = (03 03 03 03 03 03 03 03)
pshufd xmm1, xmm7, 0x00 ; xmm1 = col4 = (04 04 04 04 04 04 04 04)
pshufd xmm4, xmm7, 0x55 ; xmm4 = col5 = (05 05 05 05 05 05 05 05)
pshufd xmm3, xmm7, 0xAA ; xmm3 = col6 = (06 06 06 06 06 06 06 06)
pshufd xmm7, xmm7, 0xFF ; xmm7 = col7 = (07 07 07 07 07 07 07 07)
movdqa XMMWORD [wk(0)], xmm2 ; wk(0) = col1
movdqa XMMWORD [wk(1)], xmm0 ; wk(1) = col3
jmp near .column_end
%endif
.columnDCT:
; -- Even part
movdqa xmm0, XMMWORD [XMMBLOCK(0, 0, rsi, SIZEOF_JCOEF)]
movdqa xmm1, XMMWORD [XMMBLOCK(2, 0, rsi, SIZEOF_JCOEF)]
pmullw xmm0, XMMWORD [XMMBLOCK(0, 0, rdx, SIZEOF_IFAST_MULT_TYPE)]
pmullw xmm1, XMMWORD [XMMBLOCK(2, 0, rdx, SIZEOF_IFAST_MULT_TYPE)]
movdqa xmm2, XMMWORD [XMMBLOCK(4, 0, rsi, SIZEOF_JCOEF)]
movdqa xmm3, XMMWORD [XMMBLOCK(6, 0, rsi, SIZEOF_JCOEF)]
pmullw xmm2, XMMWORD [XMMBLOCK(4, 0, rdx, SIZEOF_IFAST_MULT_TYPE)]
pmullw xmm3, XMMWORD [XMMBLOCK(6, 0, rdx, SIZEOF_IFAST_MULT_TYPE)]
movdqa xmm4, xmm0
movdqa xmm5, xmm1
psubw xmm0, xmm2 ; xmm0 = tmp11
psubw xmm1, xmm3
paddw xmm4, xmm2 ; xmm4 = tmp10
paddw xmm5, xmm3 ; xmm5 = tmp13
psllw xmm1, PRE_MULTIPLY_SCALE_BITS
pmulhw xmm1, [rel PW_F1414]
psubw xmm1, xmm5 ; xmm1 = tmp12
movdqa xmm6, xmm4
movdqa xmm7, xmm0
psubw xmm4, xmm5 ; xmm4 = tmp3
psubw xmm0, xmm1 ; xmm0 = tmp2
paddw xmm6, xmm5 ; xmm6 = tmp0
paddw xmm7, xmm1 ; xmm7 = tmp1
movdqa XMMWORD [wk(1)], xmm4 ; wk(1) = tmp3
movdqa XMMWORD [wk(0)], xmm0 ; wk(0) = tmp2
; -- Odd part
movdqa xmm2, XMMWORD [XMMBLOCK(1, 0, rsi, SIZEOF_JCOEF)]
movdqa xmm3, XMMWORD [XMMBLOCK(3, 0, rsi, SIZEOF_JCOEF)]
pmullw xmm2, XMMWORD [XMMBLOCK(1, 0, rdx, SIZEOF_IFAST_MULT_TYPE)]
pmullw xmm3, XMMWORD [XMMBLOCK(3, 0, rdx, SIZEOF_IFAST_MULT_TYPE)]
movdqa xmm5, XMMWORD [XMMBLOCK(5, 0, rsi, SIZEOF_JCOEF)]
movdqa xmm1, XMMWORD [XMMBLOCK(7, 0, rsi, SIZEOF_JCOEF)]
pmullw xmm5, XMMWORD [XMMBLOCK(5, 0, rdx, SIZEOF_IFAST_MULT_TYPE)]
pmullw xmm1, XMMWORD [XMMBLOCK(7, 0, rdx, SIZEOF_IFAST_MULT_TYPE)]
movdqa xmm4, xmm2
movdqa xmm0, xmm5
psubw xmm2, xmm1 ; xmm2 = z12
psubw xmm5, xmm3 ; xmm5 = z10
paddw xmm4, xmm1 ; xmm4 = z11
paddw xmm0, xmm3 ; xmm0 = z13
movdqa xmm1, xmm5 ; xmm1 = z10(unscaled)
psllw xmm2, PRE_MULTIPLY_SCALE_BITS
psllw xmm5, PRE_MULTIPLY_SCALE_BITS
movdqa xmm3, xmm4
psubw xmm4, xmm0
paddw xmm3, xmm0 ; xmm3 = tmp7
psllw xmm4, PRE_MULTIPLY_SCALE_BITS
pmulhw xmm4, [rel PW_F1414] ; xmm4 = tmp11
; To avoid overflow...
;
; (Original)
; tmp12 = -2.613125930 * z10 + z5;
;
; (This implementation)
; tmp12 = (-1.613125930 - 1) * z10 + z5;
; = -1.613125930 * z10 - z10 + z5;
movdqa xmm0, xmm5
paddw xmm5, xmm2
pmulhw xmm5, [rel PW_F1847] ; xmm5 = z5
pmulhw xmm0, [rel PW_MF1613]
pmulhw xmm2, [rel PW_F1082]
psubw xmm0, xmm1
psubw xmm2, xmm5 ; xmm2 = tmp10
paddw xmm0, xmm5 ; xmm0 = tmp12
; -- Final output stage
psubw xmm0, xmm3 ; xmm0 = tmp6
movdqa xmm1, xmm6
movdqa xmm5, xmm7
paddw xmm6, xmm3 ; xmm6 = data0 = (00 01 02 03 04 05 06 07)
paddw xmm7, xmm0 ; xmm7 = data1 = (10 11 12 13 14 15 16 17)
psubw xmm1, xmm3 ; xmm1 = data7 = (70 71 72 73 74 75 76 77)
psubw xmm5, xmm0 ; xmm5 = data6 = (60 61 62 63 64 65 66 67)
psubw xmm4, xmm0 ; xmm4 = tmp5
movdqa xmm3, xmm6 ; transpose coefficients(phase 1)
punpcklwd xmm6, xmm7 ; xmm6 = (00 10 01 11 02 12 03 13)
punpckhwd xmm3, xmm7 ; xmm3 = (04 14 05 15 06 16 07 17)
movdqa xmm0, xmm5 ; transpose coefficients(phase 1)
punpcklwd xmm5, xmm1 ; xmm5 = (60 70 61 71 62 72 63 73)
punpckhwd xmm0, xmm1 ; xmm0 = (64 74 65 75 66 76 67 77)
movdqa xmm7, XMMWORD [wk(0)] ; xmm7 = tmp2
movdqa xmm1, XMMWORD [wk(1)] ; xmm1 = tmp3
movdqa XMMWORD [wk(0)], xmm5 ; wk(0) = (60 70 61 71 62 72 63 73)
movdqa XMMWORD [wk(1)], xmm0 ; wk(1) = (64 74 65 75 66 76 67 77)
paddw xmm2, xmm4 ; xmm2 = tmp4
movdqa xmm5, xmm7
movdqa xmm0, xmm1
paddw xmm7, xmm4 ; xmm7 = data2 = (20 21 22 23 24 25 26 27)
paddw xmm1, xmm2 ; xmm1 = data4 = (40 41 42 43 44 45 46 47)
psubw xmm5, xmm4 ; xmm5 = data5 = (50 51 52 53 54 55 56 57)
psubw xmm0, xmm2 ; xmm0 = data3 = (30 31 32 33 34 35 36 37)
movdqa xmm4, xmm7 ; transpose coefficients(phase 1)
punpcklwd xmm7, xmm0 ; xmm7 = (20 30 21 31 22 32 23 33)
punpckhwd xmm4, xmm0 ; xmm4 = (24 34 25 35 26 36 27 37)
movdqa xmm2, xmm1 ; transpose coefficients(phase 1)
punpcklwd xmm1, xmm5 ; xmm1 = (40 50 41 51 42 52 43 53)
punpckhwd xmm2, xmm5 ; xmm2 = (44 54 45 55 46 56 47 57)
movdqa xmm0, xmm3 ; transpose coefficients(phase 2)
punpckldq xmm3, xmm4 ; xmm3 = (04 14 24 34 05 15 25 35)
punpckhdq xmm0, xmm4 ; xmm0 = (06 16 26 36 07 17 27 37)
movdqa xmm5, xmm6 ; transpose coefficients(phase 2)
punpckldq xmm6, xmm7 ; xmm6 = (00 10 20 30 01 11 21 31)
punpckhdq xmm5, xmm7 ; xmm5 = (02 12 22 32 03 13 23 33)
movdqa xmm4, XMMWORD [wk(0)] ; xmm4 = (60 70 61 71 62 72 63 73)
movdqa xmm7, XMMWORD [wk(1)] ; xmm7 = (64 74 65 75 66 76 67 77)
movdqa XMMWORD [wk(0)], xmm3 ; wk(0) = (04 14 24 34 05 15 25 35)
movdqa XMMWORD [wk(1)], xmm0 ; wk(1) = (06 16 26 36 07 17 27 37)
movdqa xmm3, xmm1 ; transpose coefficients(phase 2)
punpckldq xmm1, xmm4 ; xmm1 = (40 50 60 70 41 51 61 71)
punpckhdq xmm3, xmm4 ; xmm3 = (42 52 62 72 43 53 63 73)
movdqa xmm0, xmm2 ; transpose coefficients(phase 2)
punpckldq xmm2, xmm7 ; xmm2 = (44 54 64 74 45 55 65 75)
punpckhdq xmm0, xmm7 ; xmm0 = (46 56 66 76 47 57 67 77)
movdqa xmm4, xmm6 ; transpose coefficients(phase 3)
punpcklqdq xmm6, xmm1 ; xmm6 = col0 = (00 10 20 30 40 50 60 70)
punpckhqdq xmm4, xmm1 ; xmm4 = col1 = (01 11 21 31 41 51 61 71)
movdqa xmm7, xmm5 ; transpose coefficients(phase 3)
punpcklqdq xmm5, xmm3 ; xmm5 = col2 = (02 12 22 32 42 52 62 72)
punpckhqdq xmm7, xmm3 ; xmm7 = col3 = (03 13 23 33 43 53 63 73)
movdqa xmm1, XMMWORD [wk(0)] ; xmm1 = (04 14 24 34 05 15 25 35)
movdqa xmm3, XMMWORD [wk(1)] ; xmm3 = (06 16 26 36 07 17 27 37)
movdqa XMMWORD [wk(0)], xmm4 ; wk(0) = col1
movdqa XMMWORD [wk(1)], xmm7 ; wk(1) = col3
movdqa xmm4, xmm1 ; transpose coefficients(phase 3)
punpcklqdq xmm1, xmm2 ; xmm1 = col4 = (04 14 24 34 44 54 64 74)
punpckhqdq xmm4, xmm2 ; xmm4 = col5 = (05 15 25 35 45 55 65 75)
movdqa xmm7, xmm3 ; transpose coefficients(phase 3)
punpcklqdq xmm3, xmm0 ; xmm3 = col6 = (06 16 26 36 46 56 66 76)
punpckhqdq xmm7, xmm0 ; xmm7 = col7 = (07 17 27 37 47 57 67 77)
.column_end:
; -- Prefetch the next coefficient block
prefetchnta [rsi + DCTSIZE2 * SIZEOF_JCOEF + 0 * 32]
prefetchnta [rsi + DCTSIZE2 * SIZEOF_JCOEF + 1 * 32]
prefetchnta [rsi + DCTSIZE2 * SIZEOF_JCOEF + 2 * 32]
prefetchnta [rsi + DCTSIZE2 * SIZEOF_JCOEF + 3 * 32]
; ---- Pass 2: process rows from work array, store into output array.
mov rdi, r12 ; (JSAMPROW *)
mov eax, r13d
; -- Even part
; xmm6 = col0, xmm5 = col2, xmm1 = col4, xmm3 = col6
movdqa xmm2, xmm6
movdqa xmm0, xmm5
psubw xmm6, xmm1 ; xmm6 = tmp11
psubw xmm5, xmm3
paddw xmm2, xmm1 ; xmm2 = tmp10
paddw xmm0, xmm3 ; xmm0 = tmp13
psllw xmm5, PRE_MULTIPLY_SCALE_BITS
pmulhw xmm5, [rel PW_F1414]
psubw xmm5, xmm0 ; xmm5 = tmp12
movdqa xmm1, xmm2
movdqa xmm3, xmm6
psubw xmm2, xmm0 ; xmm2 = tmp3
psubw xmm6, xmm5 ; xmm6 = tmp2
paddw xmm1, xmm0 ; xmm1 = tmp0
paddw xmm3, xmm5 ; xmm3 = tmp1
movdqa xmm0, XMMWORD [wk(0)] ; xmm0 = col1
movdqa xmm5, XMMWORD [wk(1)] ; xmm5 = col3
movdqa XMMWORD [wk(0)], xmm2 ; wk(0) = tmp3
movdqa XMMWORD [wk(1)], xmm6 ; wk(1) = tmp2
; -- Odd part
; xmm0 = col1, xmm5 = col3, xmm4 = col5, xmm7 = col7
movdqa xmm2, xmm0
movdqa xmm6, xmm4
psubw xmm0, xmm7 ; xmm0 = z12
psubw xmm4, xmm5 ; xmm4 = z10
paddw xmm2, xmm7 ; xmm2 = z11
paddw xmm6, xmm5 ; xmm6 = z13
movdqa xmm7, xmm4 ; xmm7 = z10(unscaled)
psllw xmm0, PRE_MULTIPLY_SCALE_BITS
psllw xmm4, PRE_MULTIPLY_SCALE_BITS
movdqa xmm5, xmm2
psubw xmm2, xmm6
paddw xmm5, xmm6 ; xmm5 = tmp7
psllw xmm2, PRE_MULTIPLY_SCALE_BITS
pmulhw xmm2, [rel PW_F1414] ; xmm2 = tmp11
; To avoid overflow...
;
; (Original)
; tmp12 = -2.613125930 * z10 + z5;
;
; (This implementation)
; tmp12 = (-1.613125930 - 1) * z10 + z5;
; = -1.613125930 * z10 - z10 + z5;
movdqa xmm6, xmm4
paddw xmm4, xmm0
pmulhw xmm4, [rel PW_F1847] ; xmm4 = z5
pmulhw xmm6, [rel PW_MF1613]
pmulhw xmm0, [rel PW_F1082]
psubw xmm6, xmm7
psubw xmm0, xmm4 ; xmm0 = tmp10
paddw xmm6, xmm4 ; xmm6 = tmp12
; -- Final output stage
psubw xmm6, xmm5 ; xmm6 = tmp6
movdqa xmm7, xmm1
movdqa xmm4, xmm3
paddw xmm1, xmm5 ; xmm1 = data0 = (00 10 20 30 40 50 60 70)
paddw xmm3, xmm6 ; xmm3 = data1 = (01 11 21 31 41 51 61 71)
psraw xmm1, (PASS1_BITS + 3) ; descale
psraw xmm3, (PASS1_BITS + 3) ; descale
psubw xmm7, xmm5 ; xmm7 = data7 = (07 17 27 37 47 57 67 77)
psubw xmm4, xmm6 ; xmm4 = data6 = (06 16 26 36 46 56 66 76)
psraw xmm7, (PASS1_BITS + 3) ; descale
psraw xmm4, (PASS1_BITS + 3) ; descale
psubw xmm2, xmm6 ; xmm2 = tmp5
packsswb xmm1, xmm4
; xmm1 = (00 10 20 30 40 50 60 70 06 16 26 36 46 56 66 76)
packsswb xmm3, xmm7
; xmm3 = (01 11 21 31 41 51 61 71 07 17 27 37 47 57 67 77)
movdqa xmm5, XMMWORD [wk(1)] ; xmm5 = tmp2
movdqa xmm6, XMMWORD [wk(0)] ; xmm6 = tmp3
paddw xmm0, xmm2 ; xmm0 = tmp4
movdqa xmm4, xmm5
movdqa xmm7, xmm6
paddw xmm5, xmm2 ; xmm5 = data2 = (02 12 22 32 42 52 62 72)
paddw xmm6, xmm0 ; xmm6 = data4 = (04 14 24 34 44 54 64 74)
psraw xmm5, (PASS1_BITS + 3) ; descale
psraw xmm6, (PASS1_BITS + 3) ; descale
psubw xmm4, xmm2 ; xmm4 = data5 = (05 15 25 35 45 55 65 75)
psubw xmm7, xmm0 ; xmm7 = data3 = (03 13 23 33 43 53 63 73)
psraw xmm4, (PASS1_BITS + 3) ; descale
psraw xmm7, (PASS1_BITS + 3) ; descale
movdqa xmm2, [rel PB_CENTERJSAMP] ; xmm2 = [rel PB_CENTERJSAMP]
packsswb xmm5, xmm6
; xmm5 = (02 12 22 32 42 52 62 72 04 14 24 34 44 54 64 74)
packsswb xmm7, xmm4
; xmm7 = (03 13 23 33 43 53 63 73 05 15 25 35 45 55 65 75)
paddb xmm1, xmm2
paddb xmm3, xmm2
paddb xmm5, xmm2
paddb xmm7, xmm2
; transpose coefficients(phase 1)
movdqa xmm0, xmm1
punpcklbw xmm1, xmm3
; xmm1 = (00 01 10 11 20 21 30 31 40 41 50 51 60 61 70 71)
punpckhbw xmm0, xmm3
; xmm0 = (06 07 16 17 26 27 36 37 46 47 56 57 66 67 76 77)
movdqa xmm6, xmm5
punpcklbw xmm5, xmm7
; xmm5 = (02 03 12 13 22 23 32 33 42 43 52 53 62 63 72 73)
punpckhbw xmm6, xmm7
; xmm6 = (04 05 14 15 24 25 34 35 44 45 54 55 64 65 74 75)
; transpose coefficients(phase 2)
movdqa xmm4, xmm1
punpcklwd xmm1, xmm5
; xmm1 = (00 01 02 03 10 11 12 13 20 21 22 23 30 31 32 33)
punpckhwd xmm4, xmm5
; xmm4 = (40 41 42 43 50 51 52 53 60 61 62 63 70 71 72 73)
movdqa xmm2, xmm6
punpcklwd xmm6, xmm0
; xmm6 = (04 05 06 07 14 15 16 17 24 25 26 27 34 35 36 37)
punpckhwd xmm2, xmm0
; xmm2 = (44 45 46 47 54 55 56 57 64 65 66 67 74 75 76 77)
; transpose coefficients(phase 3)
movdqa xmm3, xmm1
punpckldq xmm1, xmm6
; xmm1 = (00 01 02 03 04 05 06 07 10 11 12 13 14 15 16 17)
punpckhdq xmm3, xmm6
; xmm3 = (20 21 22 23 24 25 26 27 30 31 32 33 34 35 36 37)
movdqa xmm7, xmm4
punpckldq xmm4, xmm2
; xmm4 = (40 41 42 43 44 45 46 47 50 51 52 53 54 55 56 57)
punpckhdq xmm7, xmm2
; xmm7 = (60 61 62 63 64 65 66 67 70 71 72 73 74 75 76 77)
pshufd xmm5, xmm1, 0x4E
; xmm5 = (10 11 12 13 14 15 16 17 00 01 02 03 04 05 06 07)
pshufd xmm0, xmm3, 0x4E
; xmm0 = (30 31 32 33 34 35 36 37 20 21 22 23 24 25 26 27)
pshufd xmm6, xmm4, 0x4E
; xmm6 = (50 51 52 53 54 55 56 57 40 41 42 43 44 45 46 47)
pshufd xmm2, xmm7, 0x4E
; xmm2 = (70 71 72 73 74 75 76 77 60 61 62 63 64 65 66 67)
mov rdxp, JSAMPROW [rdi + 0 * SIZEOF_JSAMPROW]
mov rsip, JSAMPROW [rdi + 2 * SIZEOF_JSAMPROW]
movq XMM_MMWORD [rdx + rax * SIZEOF_JSAMPLE], xmm1
movq XMM_MMWORD [rsi + rax * SIZEOF_JSAMPLE], xmm3
mov rdxp, JSAMPROW [rdi + 4 * SIZEOF_JSAMPROW]
mov rsip, JSAMPROW [rdi + 6 * SIZEOF_JSAMPROW]
movq XMM_MMWORD [rdx + rax * SIZEOF_JSAMPLE], xmm4
movq XMM_MMWORD [rsi + rax * SIZEOF_JSAMPLE], xmm7
mov rdxp, JSAMPROW [rdi + 1 * SIZEOF_JSAMPROW]
mov rsip, JSAMPROW [rdi + 3 * SIZEOF_JSAMPROW]
movq XMM_MMWORD [rdx + rax * SIZEOF_JSAMPLE], xmm5
movq XMM_MMWORD [rsi + rax * SIZEOF_JSAMPLE], xmm0
mov rdxp, JSAMPROW [rdi + 5 * SIZEOF_JSAMPROW]
mov rsip, JSAMPROW [rdi + 7 * SIZEOF_JSAMPROW]
movq XMM_MMWORD [rdx + rax * SIZEOF_JSAMPLE], xmm6
movq XMM_MMWORD [rsi + rax * SIZEOF_JSAMPLE], xmm2
UNCOLLECT_ARGS 4
lea rsp, [rbp - 8]
pop r15
pop rbp
ret
ret
; For some reason, the OS X linker does not honor the request to align the
; segment unless we do this.
align 32
lib/libjpeg-turbo/simd/x86_64/jidctint-avx2.asm
Executable
+426
View File
@@ -0,0 +1,426 @@
;
; Accurate integer IDCT (64-bit AVX2)
;
; Copyright 2009 Pierre Ossman <[email protected]> for Cendio AB
; Copyright (C) 2009, 2016, 2018, 2020, 2024-2025, D. R. Commander.
; Copyright (C) 2018, Matthias Räncker.
;
; Based on the x86 SIMD extension for IJG JPEG library
; Copyright (C) 1999-2006, MIYASAKA Masaru.
; For conditions of distribution and use, see copyright notice in jsimdext.inc
;
; This file should be assembled with NASM (Netwide Assembler) or Yasm.
;
; This file contains a slower but more accurate integer implementation of the
; inverse DCT (Discrete Cosine Transform). The following code is based
; directly on the IJG's original jidctint.c; see jidctint.c for more details.
%include "jsimdext.inc"
%include "jdct.inc"
; --------------------------------------------------------------------------
%define CONST_BITS 13
%define PASS1_BITS 2
%define DESCALE_P1 (CONST_BITS - PASS1_BITS)
%define DESCALE_P2 (CONST_BITS + PASS1_BITS + 3)
%if CONST_BITS == 13
F_0_298 equ 2446 ; FIX(0.298631336)
F_0_390 equ 3196 ; FIX(0.390180644)
F_0_541 equ 4433 ; FIX(0.541196100)
F_0_765 equ 6270 ; FIX(0.765366865)
F_0_899 equ 7373 ; FIX(0.899976223)
F_1_175 equ 9633 ; FIX(1.175875602)
F_1_501 equ 12299 ; FIX(1.501321110)
F_1_847 equ 15137 ; FIX(1.847759065)
F_1_961 equ 16069 ; FIX(1.961570560)
F_2_053 equ 16819 ; FIX(2.053119869)
F_2_562 equ 20995 ; FIX(2.562915447)
F_3_072 equ 25172 ; FIX(3.072711026)
%else
; NASM cannot do compile-time arithmetic on floating-point constants.
%define DESCALE(x, n) (((x) + (1 << ((n) - 1))) >> (n))
F_0_298 equ DESCALE( 320652955, 30 - CONST_BITS) ; FIX(0.298631336)
F_0_390 equ DESCALE( 418953276, 30 - CONST_BITS) ; FIX(0.390180644)
F_0_541 equ DESCALE( 581104887, 30 - CONST_BITS) ; FIX(0.541196100)
F_0_765 equ DESCALE( 821806413, 30 - CONST_BITS) ; FIX(0.765366865)
F_0_899 equ DESCALE( 966342111, 30 - CONST_BITS) ; FIX(0.899976223)
F_1_175 equ DESCALE(1262586813, 30 - CONST_BITS) ; FIX(1.175875602)
F_1_501 equ DESCALE(1612031267, 30 - CONST_BITS) ; FIX(1.501321110)
F_1_847 equ DESCALE(1984016188, 30 - CONST_BITS) ; FIX(1.847759065)
F_1_961 equ DESCALE(2106220350, 30 - CONST_BITS) ; FIX(1.961570560)
F_2_053 equ DESCALE(2204520673, 30 - CONST_BITS) ; FIX(2.053119869)
F_2_562 equ DESCALE(2751909506, 30 - CONST_BITS) ; FIX(2.562915447)
F_3_072 equ DESCALE(3299298341, 30 - CONST_BITS) ; FIX(3.072711026)
%endif
; --------------------------------------------------------------------------
; In-place 8x8x16-bit inverse matrix transpose using AVX2 instructions
; %1-%4: Input/output registers
; %5-%8: Temp registers
%macro DOTRANSPOSE 8
; %5 = (00 10 20 30 40 50 60 70 01 11 21 31 41 51 61 71)
; %6 = (03 13 23 33 43 53 63 73 02 12 22 32 42 52 62 72)
; %7 = (04 14 24 34 44 54 64 74 05 15 25 35 45 55 65 75)
; %8 = (07 17 27 37 47 57 67 77 06 16 26 36 46 56 66 76)
; transpose coefficients(phase 1)
vpermq %5, %1, 0xD8
; %5 = (00 10 20 30 01 11 21 31 40 50 60 70 41 51 61 71)
vpermq %6, %2, 0x72
; %6 = (02 12 22 32 03 13 23 33 42 52 62 72 43 53 63 73)
vpermq %7, %3, 0xD8
; %7 = (04 14 24 34 05 15 25 35 44 54 64 74 45 55 65 75)
vpermq %8, %4, 0x72
; %8 = (06 16 26 36 07 17 27 37 46 56 66 76 47 57 67 77)
; transpose coefficients(phase 2)
vpunpcklwd %1, %5, %6
; %1 = (00 02 10 12 20 22 30 32 40 42 50 52 60 62 70 72)
vpunpckhwd %2, %5, %6
; %2 = (01 03 11 13 21 23 31 33 41 43 51 53 61 63 71 73)
vpunpcklwd %3, %7, %8
; %3 = (04 06 14 16 24 26 34 36 44 46 54 56 64 66 74 76)
vpunpckhwd %4, %7, %8
; %4 = (05 07 15 17 25 27 35 37 45 47 55 57 65 67 75 77)
; transpose coefficients(phase 3)
vpunpcklwd %5, %1, %2
; %5 = (00 01 02 03 10 11 12 13 40 41 42 43 50 51 52 53)
vpunpcklwd %6, %3, %4
; %6 = (04 05 06 07 14 15 16 17 44 45 46 47 54 55 56 57)
vpunpckhwd %7, %1, %2
; %7 = (20 21 22 23 30 31 32 33 60 61 62 63 70 71 72 73)
vpunpckhwd %8, %3, %4
; %8 = (24 25 26 27 34 35 36 37 64 65 66 67 74 75 76 77)
; transpose coefficients(phase 4)
vpunpcklqdq %1, %5, %6
; %1 = (00 01 02 03 04 05 06 07 40 41 42 43 44 45 46 47)
vpunpckhqdq %2, %5, %6
; %2 = (10 11 12 13 14 15 16 17 50 51 52 53 54 55 56 57)
vpunpcklqdq %3, %7, %8
; %3 = (20 21 22 23 24 25 26 27 60 61 62 63 64 65 66 67)
vpunpckhqdq %4, %7, %8
; %4 = (30 31 32 33 34 35 36 37 70 71 72 73 74 75 76 77)
%endmacro
; --------------------------------------------------------------------------
; In-place 8x8x16-bit accurate integer inverse DCT using AVX2 instructions
; %1-%4: Input/output registers
; %5-%12: Temp registers
; %9: Pass (1 or 2)
%macro DODCT 13
; -- Even part
; (Original)
; z1 = (z2 + z3) * 0.541196100;
; tmp2 = z1 + z3 * -1.847759065;
; tmp3 = z1 + z2 * 0.765366865;
;
; (This implementation)
; tmp2 = z2 * 0.541196100 + z3 * (0.541196100 - 1.847759065);
; tmp3 = z2 * (0.541196100 + 0.765366865) + z3 * 0.541196100;
vperm2i128 %6, %3, %3, 0x01 ; %6 = in6_2
vpunpcklwd %5, %3, %6 ; %5 = in26_62L
vpunpckhwd %6, %3, %6 ; %6 = in26_62H
vpmaddwd %5, %5, [rel PW_F130_F054_MF130_F054] ; %5 = tmp3_2L
vpmaddwd %6, %6, [rel PW_F130_F054_MF130_F054] ; %6 = tmp3_2H
vperm2i128 %7, %1, %1, 0x01 ; %7 = in4_0
vpsignw %1, %1, [rel PW_1_NEG1]
vpaddw %7, %7, %1 ; %7 = (in0 + in4)_(in0 - in4)
vpxor %1, %1, %1
vpunpcklwd %8, %1, %7 ; %8 = tmp0_1L
vpunpckhwd %1, %1, %7 ; %1 = tmp0_1H
vpsrad %8, %8, (16 - CONST_BITS)
; vpsrad %8, 16 & vpslld %8, CONST_BITS
vpsrad %1, %1, (16 - CONST_BITS)
; vpsrad %1, 16 & vpslld %1, CONST_BITS
vpsubd %11, %8, %5 ; %11 = tmp0_1L - tmp3_2L = tmp13_12L
vpaddd %9, %8, %5 ; %9 = tmp0_1L + tmp3_2L = tmp10_11L
vpsubd %12, %1, %6 ; %12 = tmp0_1H - tmp3_2H = tmp13_12H
vpaddd %10, %1, %6 ; %10 = tmp0_1H + tmp3_2H = tmp10_11H
; -- Odd part
vpaddw %1, %4, %2 ; %1 = in7_5 + in3_1 = z3_4
; (Original)
; z5 = (z3 + z4) * 1.175875602;
; z3 = z3 * -1.961570560; z4 = z4 * -0.390180644;
; z3 += z5; z4 += z5;
;
; (This implementation)
; z3 = z3 * (1.175875602 - 1.961570560) + z4 * 1.175875602;
; z4 = z3 * 1.175875602 + z4 * (1.175875602 - 0.390180644);
vperm2i128 %8, %1, %1, 0x01 ; %8 = z4_3
vpunpcklwd %7, %1, %8 ; %7 = z34_43L
vpunpckhwd %8, %1, %8 ; %8 = z34_43H
vpmaddwd %7, %7, [rel PW_MF078_F117_F078_F117] ; %7 = z3_4L
vpmaddwd %8, %8, [rel PW_MF078_F117_F078_F117] ; %8 = z3_4H
; (Original)
; z1 = tmp0 + tmp3; z2 = tmp1 + tmp2;
; tmp0 = tmp0 * 0.298631336; tmp1 = tmp1 * 2.053119869;
; tmp2 = tmp2 * 3.072711026; tmp3 = tmp3 * 1.501321110;
; z1 = z1 * -0.899976223; z2 = z2 * -2.562915447;
; tmp0 += z1 + z3; tmp1 += z2 + z4;
; tmp2 += z2 + z3; tmp3 += z1 + z4;
;
; (This implementation)
; tmp0 = tmp0 * (0.298631336 - 0.899976223) + tmp3 * -0.899976223;
; tmp1 = tmp1 * (2.053119869 - 2.562915447) + tmp2 * -2.562915447;
; tmp2 = tmp1 * -2.562915447 + tmp2 * (3.072711026 - 2.562915447);
; tmp3 = tmp0 * -0.899976223 + tmp3 * (1.501321110 - 0.899976223);
; tmp0 += z3; tmp1 += z4;
; tmp2 += z3; tmp3 += z4;
vperm2i128 %2, %2, %2, 0x01 ; %2 = in1_3
vpunpcklwd %3, %4, %2 ; %3 = in71_53L
vpunpckhwd %4, %4, %2 ; %4 = in71_53H
vpmaddwd %5, %3, [rel PW_MF060_MF089_MF050_MF256] ; %5 = tmp0_1L
vpmaddwd %6, %4, [rel PW_MF060_MF089_MF050_MF256] ; %6 = tmp0_1H
vpaddd %5, %5, %7 ; %5 = tmp0_1L + z3_4L = tmp0_1L
vpaddd %6, %6, %8 ; %6 = tmp0_1H + z3_4H = tmp0_1H
vpmaddwd %3, %3, [rel PW_MF089_F060_MF256_F050] ; %3 = tmp3_2L
vpmaddwd %4, %4, [rel PW_MF089_F060_MF256_F050] ; %4 = tmp3_2H
vperm2i128 %7, %7, %7, 0x01 ; %7 = z4_3L
vperm2i128 %8, %8, %8, 0x01 ; %8 = z4_3H
vpaddd %7, %3, %7 ; %7 = tmp3_2L + z4_3L = tmp3_2L
vpaddd %8, %4, %8 ; %8 = tmp3_2H + z4_3H = tmp3_2H
; -- Final output stage
vpaddd %1, %9, %7 ; %1 = tmp10_11L + tmp3_2L = data0_1L
vpaddd %2, %10, %8 ; %2 = tmp10_11H + tmp3_2H = data0_1H
vpaddd %1, %1, [rel PD_DESCALE_P %+ %13]
vpaddd %2, %2, [rel PD_DESCALE_P %+ %13]
vpsrad %1, %1, DESCALE_P %+ %13
vpsrad %2, %2, DESCALE_P %+ %13
vpackssdw %1, %1, %2 ; %1 = data0_1
vpsubd %3, %9, %7 ; %3 = tmp10_11L - tmp3_2L = data7_6L
vpsubd %4, %10, %8 ; %4 = tmp10_11H - tmp3_2H = data7_6H
vpaddd %3, %3, [rel PD_DESCALE_P %+ %13]
vpaddd %4, %4, [rel PD_DESCALE_P %+ %13]
vpsrad %3, %3, DESCALE_P %+ %13
vpsrad %4, %4, DESCALE_P %+ %13
vpackssdw %4, %3, %4 ; %4 = data7_6
vpaddd %7, %11, %5 ; %7 = tmp13_12L + tmp0_1L = data3_2L
vpaddd %8, %12, %6 ; %8 = tmp13_12H + tmp0_1H = data3_2H
vpaddd %7, %7, [rel PD_DESCALE_P %+ %13]
vpaddd %8, %8, [rel PD_DESCALE_P %+ %13]
vpsrad %7, %7, DESCALE_P %+ %13
vpsrad %8, %8, DESCALE_P %+ %13
vpackssdw %2, %7, %8 ; %2 = data3_2
vpsubd %7, %11, %5 ; %7 = tmp13_12L - tmp0_1L = data4_5L
vpsubd %8, %12, %6 ; %8 = tmp13_12H - tmp0_1H = data4_5H
vpaddd %7, %7, [rel PD_DESCALE_P %+ %13]
vpaddd %8, %8, [rel PD_DESCALE_P %+ %13]
vpsrad %7, %7, DESCALE_P %+ %13
vpsrad %8, %8, DESCALE_P %+ %13
vpackssdw %3, %7, %8 ; %3 = data4_5
%endmacro
; --------------------------------------------------------------------------
SECTION SEG_CONST
ALIGNZ 32
GLOBAL_DATA(jconst_idct_islow_avx2)
EXTN(jconst_idct_islow_avx2):
PW_F130_F054_MF130_F054 times 4 dw (F_0_541 + F_0_765), F_0_541
times 4 dw (F_0_541 - F_1_847), F_0_541
PW_MF078_F117_F078_F117 times 4 dw (F_1_175 - F_1_961), F_1_175
times 4 dw (F_1_175 - F_0_390), F_1_175
PW_MF060_MF089_MF050_MF256 times 4 dw (F_0_298 - F_0_899), -F_0_899
times 4 dw (F_2_053 - F_2_562), -F_2_562
PW_MF089_F060_MF256_F050 times 4 dw -F_0_899, (F_1_501 - F_0_899)
times 4 dw -F_2_562, (F_3_072 - F_2_562)
PD_DESCALE_P1 times 8 dd 1 << (DESCALE_P1 - 1)
PD_DESCALE_P2 times 8 dd 1 << (DESCALE_P2 - 1)
PB_CENTERJSAMP times 32 db CENTERJSAMPLE
PW_1_NEG1 times 8 dw 1
times 8 dw -1
ALIGNZ 32
; --------------------------------------------------------------------------
SECTION SEG_TEXT
BITS 64
; Perform dequantization and inverse DCT on one block of coefficients.
;
; GLOBAL(void)
; jsimd_idct_islow_avx2(void *dct_table, JCOEFPTR coef_block,
; JSAMPARRAY output_buf, JDIMENSION output_col)
;
; r10 = jpeg_component_info *compptr
; r11 = JCOEFPTR coef_block
; r12 = JSAMPARRAY output_buf
; r13d = JDIMENSION output_col
align 32
GLOBAL_FUNCTION(jsimd_idct_islow_avx2)
EXTN(jsimd_idct_islow_avx2):
ENDBR64
push rbp
mov rbp, rsp ; rbp = aligned rbp
PUSH_XMM 4
COLLECT_ARGS 4
; ---- Pass 1: process columns.
%ifndef NO_ZERO_COLUMN_TEST_ISLOW_AVX2
mov eax, dword [DWBLOCK(1, 0, r11, SIZEOF_JCOEF)]
or eax, dword [DWBLOCK(2, 0, r11, SIZEOF_JCOEF)]
jnz near .columnDCT
movdqa xmm0, XMMWORD [XMMBLOCK(1, 0, r11, SIZEOF_JCOEF)]
movdqa xmm1, XMMWORD [XMMBLOCK(2, 0, r11, SIZEOF_JCOEF)]
vpor xmm0, xmm0, XMMWORD [XMMBLOCK(3, 0, r11, SIZEOF_JCOEF)]
vpor xmm1, xmm1, XMMWORD [XMMBLOCK(4, 0, r11, SIZEOF_JCOEF)]
vpor xmm0, xmm0, XMMWORD [XMMBLOCK(5, 0, r11, SIZEOF_JCOEF)]
vpor xmm1, xmm1, XMMWORD [XMMBLOCK(6, 0, r11, SIZEOF_JCOEF)]
vpor xmm0, xmm0, XMMWORD [XMMBLOCK(7, 0, r11, SIZEOF_JCOEF)]
vpor xmm1, xmm1, xmm0
vpacksswb xmm1, xmm1, xmm1
vpacksswb xmm1, xmm1, xmm1
movd eax, xmm1
test rax, rax
jnz short .columnDCT
; -- AC terms all zero
movdqa xmm5, XMMWORD [XMMBLOCK(0, 0, r11, SIZEOF_JCOEF)]
vpmullw xmm5, xmm5, XMMWORD [XMMBLOCK(0, 0, r10, SIZEOF_ISLOW_MULT_TYPE)]
vpsllw xmm5, xmm5, PASS1_BITS
vpunpcklwd xmm4, xmm5, xmm5 ; xmm4 = (00 00 01 01 02 02 03 03)
vpunpckhwd xmm5, xmm5, xmm5 ; xmm5 = (04 04 05 05 06 06 07 07)
vinserti128 ymm4, ymm4, xmm5, 1
vpshufd ymm0, ymm4, 0x00
; ymm0 = col0_4 = (00 00 00 00 00 00 00 00 04 04 04 04 04 04 04 04)
vpshufd ymm1, ymm4, 0x55
; ymm1 = col1_5 = (01 01 01 01 01 01 01 01 05 05 05 05 05 05 05 05)
vpshufd ymm2, ymm4, 0xAA
; ymm2 = col2_6 = (02 02 02 02 02 02 02 02 06 06 06 06 06 06 06 06)
vpshufd ymm3, ymm4, 0xFF
; ymm3 = col3_7 = (03 03 03 03 03 03 03 03 07 07 07 07 07 07 07 07)
jmp near .column_end
%endif
.columnDCT:
vmovdqu ymm4, YMMWORD [YMMBLOCK(0, 0, r11, SIZEOF_JCOEF)]
; ymm4 = in0_1
vmovdqu ymm5, YMMWORD [YMMBLOCK(2, 0, r11, SIZEOF_JCOEF)]
; ymm5 = in2_3
vmovdqu ymm6, YMMWORD [YMMBLOCK(4, 0, r11, SIZEOF_JCOEF)]
; ymm6 = in4_5
vmovdqu ymm7, YMMWORD [YMMBLOCK(6, 0, r11, SIZEOF_JCOEF)]
; ymm7 = in6_7
vpmullw ymm4, ymm4, YMMWORD [YMMBLOCK(0, 0, r10, SIZEOF_ISLOW_MULT_TYPE)]
vpmullw ymm5, ymm5, YMMWORD [YMMBLOCK(2, 0, r10, SIZEOF_ISLOW_MULT_TYPE)]
vpmullw ymm6, ymm6, YMMWORD [YMMBLOCK(4, 0, r10, SIZEOF_ISLOW_MULT_TYPE)]
vpmullw ymm7, ymm7, YMMWORD [YMMBLOCK(6, 0, r10, SIZEOF_ISLOW_MULT_TYPE)]
vperm2i128 ymm0, ymm4, ymm6, 0x20 ; ymm0 = in0_4
vperm2i128 ymm1, ymm5, ymm4, 0x31 ; ymm1 = in3_1
vperm2i128 ymm2, ymm5, ymm7, 0x20 ; ymm2 = in2_6
vperm2i128 ymm3, ymm7, ymm6, 0x31 ; ymm3 = in7_5
DODCT ymm0, ymm1, ymm2, ymm3, ymm4, ymm5, ymm6, ymm7, ymm8, ymm9, ymm10, ymm11, 1
; ymm0 = data0_1, ymm1 = data3_2, ymm2 = data4_5, ymm3 = data7_6
DOTRANSPOSE ymm0, ymm1, ymm2, ymm3, ymm4, ymm5, ymm6, ymm7
; ymm0 = data0_4, ymm1 = data1_5, ymm2 = data2_6, ymm3 = data3_7
.column_end:
; -- Prefetch the next coefficient block
prefetchnta [r11 + DCTSIZE2 * SIZEOF_JCOEF + 0 * 32]
prefetchnta [r11 + DCTSIZE2 * SIZEOF_JCOEF + 1 * 32]
prefetchnta [r11 + DCTSIZE2 * SIZEOF_JCOEF + 2 * 32]
prefetchnta [r11 + DCTSIZE2 * SIZEOF_JCOEF + 3 * 32]
; ---- Pass 2: process rows.
vperm2i128 ymm4, ymm3, ymm1, 0x31 ; ymm3 = in7_5
vperm2i128 ymm1, ymm3, ymm1, 0x20 ; ymm1 = in3_1
DODCT ymm0, ymm1, ymm2, ymm4, ymm3, ymm5, ymm6, ymm7, ymm8, ymm9, ymm10, ymm11, 2
; ymm0 = data0_1, ymm1 = data3_2, ymm2 = data4_5, ymm4 = data7_6
DOTRANSPOSE ymm0, ymm1, ymm2, ymm4, ymm3, ymm5, ymm6, ymm7
; ymm0 = data0_4, ymm1 = data1_5, ymm2 = data2_6, ymm4 = data3_7
vpacksswb ymm0, ymm0, ymm1 ; ymm0 = data01_45
vpacksswb ymm1, ymm2, ymm4 ; ymm1 = data23_67
vpaddb ymm0, ymm0, [rel PB_CENTERJSAMP]
vpaddb ymm1, ymm1, [rel PB_CENTERJSAMP]
vextracti128 xmm6, ymm1, 1 ; xmm3 = data67
vextracti128 xmm4, ymm0, 1 ; xmm2 = data45
vextracti128 xmm2, ymm1, 0 ; xmm1 = data23
vextracti128 xmm0, ymm0, 0 ; xmm0 = data01
vpshufd xmm1, xmm0, 0x4E
; xmm1 = (10 11 12 13 14 15 16 17 00 01 02 03 04 05 06 07)
vpshufd xmm3, xmm2, 0x4E
; xmm3 = (30 31 32 33 34 35 36 37 20 21 22 23 24 25 26 27)
vpshufd xmm5, xmm4, 0x4E
; xmm5 = (50 51 52 53 54 55 56 57 40 41 42 43 44 45 46 47)
vpshufd xmm7, xmm6, 0x4E
; xmm7 = (70 71 72 73 74 75 76 77 60 61 62 63 64 65 66 67)
vzeroupper
mov eax, r13d
mov rdxp, JSAMPROW [r12 + 0 * SIZEOF_JSAMPROW] ; (JSAMPLE *)
mov rsip, JSAMPROW [r12 + 1 * SIZEOF_JSAMPROW] ; (JSAMPLE *)
movq XMM_MMWORD [rdx + rax * SIZEOF_JSAMPLE], xmm0
movq XMM_MMWORD [rsi + rax * SIZEOF_JSAMPLE], xmm1
mov rdxp, JSAMPROW [r12 + 2 * SIZEOF_JSAMPROW] ; (JSAMPLE *)
mov rsip, JSAMPROW [r12 + 3 * SIZEOF_JSAMPROW] ; (JSAMPLE *)
movq XMM_MMWORD [rdx + rax * SIZEOF_JSAMPLE], xmm2
movq XMM_MMWORD [rsi + rax * SIZEOF_JSAMPLE], xmm3
mov rdxp, JSAMPROW [r12 + 4 * SIZEOF_JSAMPROW] ; (JSAMPLE *)
mov rsip, JSAMPROW [r12 + 5 * SIZEOF_JSAMPROW] ; (JSAMPLE *)
movq XMM_MMWORD [rdx + rax * SIZEOF_JSAMPLE], xmm4
movq XMM_MMWORD [rsi + rax * SIZEOF_JSAMPLE], xmm5
mov rdxp, JSAMPROW [r12 + 6 * SIZEOF_JSAMPROW] ; (JSAMPLE *)
mov rsip, JSAMPROW [r12 + 7 * SIZEOF_JSAMPROW] ; (JSAMPLE *)
movq XMM_MMWORD [rdx + rax * SIZEOF_JSAMPLE], xmm6
movq XMM_MMWORD [rsi + rax * SIZEOF_JSAMPLE], xmm7
UNCOLLECT_ARGS 4
POP_XMM 4
pop rbp
ret
; For some reason, the OS X linker does not honor the request to align the
; segment unless we do this.
align 32
lib/libjpeg-turbo/simd/x86_64/jidctint-sse2.asm
Executable
+876
View File
@@ -0,0 +1,876 @@
;
; Accurate integer IDCT (64-bit SSE2)
;
; Copyright 2009 Pierre Ossman <[email protected]> for Cendio AB
; Copyright (C) 2009, 2016, 2020, 2024-2025, D. R. Commander.
; Copyright (C) 2018, Matthias Räncker.
; Copyright (C) 2023, Aliaksiej Kandracienka.
;
; Based on the x86 SIMD extension for IJG JPEG library
; Copyright (C) 1999-2006, MIYASAKA Masaru.
; For conditions of distribution and use, see copyright notice in jsimdext.inc
;
; This file should be assembled with NASM (Netwide Assembler) or Yasm.
;
; This file contains a slower but more accurate integer implementation of the
; inverse DCT (Discrete Cosine Transform). The following code is based
; directly on the IJG's original jidctint.c; see jidctint.c for more details.
%include "jsimdext.inc"
%include "jdct.inc"
; --------------------------------------------------------------------------
%define CONST_BITS 13
%define PASS1_BITS 2
%define DESCALE_P1 (CONST_BITS - PASS1_BITS)
%define DESCALE_P2 (CONST_BITS + PASS1_BITS + 3)
%if CONST_BITS == 13
F_0_298 equ 2446 ; FIX(0.298631336)
F_0_390 equ 3196 ; FIX(0.390180644)
F_0_541 equ 4433 ; FIX(0.541196100)
F_0_765 equ 6270 ; FIX(0.765366865)
F_0_899 equ 7373 ; FIX(0.899976223)
F_1_175 equ 9633 ; FIX(1.175875602)
F_1_501 equ 12299 ; FIX(1.501321110)
F_1_847 equ 15137 ; FIX(1.847759065)
F_1_961 equ 16069 ; FIX(1.961570560)
F_2_053 equ 16819 ; FIX(2.053119869)
F_2_562 equ 20995 ; FIX(2.562915447)
F_3_072 equ 25172 ; FIX(3.072711026)
%else
; NASM cannot do compile-time arithmetic on floating-point constants.
%define DESCALE(x, n) (((x) + (1 << ((n) - 1))) >> (n))
F_0_298 equ DESCALE( 320652955, 30 - CONST_BITS) ; FIX(0.298631336)
F_0_390 equ DESCALE( 418953276, 30 - CONST_BITS) ; FIX(0.390180644)
F_0_541 equ DESCALE( 581104887, 30 - CONST_BITS) ; FIX(0.541196100)
F_0_765 equ DESCALE( 821806413, 30 - CONST_BITS) ; FIX(0.765366865)
F_0_899 equ DESCALE( 966342111, 30 - CONST_BITS) ; FIX(0.899976223)
F_1_175 equ DESCALE(1262586813, 30 - CONST_BITS) ; FIX(1.175875602)
F_1_501 equ DESCALE(1612031267, 30 - CONST_BITS) ; FIX(1.501321110)
F_1_847 equ DESCALE(1984016188, 30 - CONST_BITS) ; FIX(1.847759065)
F_1_961 equ DESCALE(2106220350, 30 - CONST_BITS) ; FIX(1.961570560)
F_2_053 equ DESCALE(2204520673, 30 - CONST_BITS) ; FIX(2.053119869)
F_2_562 equ DESCALE(2751909506, 30 - CONST_BITS) ; FIX(2.562915447)
F_3_072 equ DESCALE(3299298341, 30 - CONST_BITS) ; FIX(3.072711026)
%endif
; --------------------------------------------------------------------------
SECTION SEG_CONST
ALIGNZ 32
GLOBAL_DATA(jconst_idct_islow_sse2)
EXTN(jconst_idct_islow_sse2):
PW_F130_F054 times 4 dw (F_0_541 + F_0_765), F_0_541
PW_F054_MF130 times 4 dw F_0_541, (F_0_541 - F_1_847)
PW_MF078_F117 times 4 dw (F_1_175 - F_1_961), F_1_175
PW_F117_F078 times 4 dw F_1_175, (F_1_175 - F_0_390)
PW_MF060_MF089 times 4 dw (F_0_298 - F_0_899), -F_0_899
PW_MF089_F060 times 4 dw -F_0_899, (F_1_501 - F_0_899)
PW_MF050_MF256 times 4 dw (F_2_053 - F_2_562), -F_2_562
PW_MF256_F050 times 4 dw -F_2_562, (F_3_072 - F_2_562)
PD_DESCALE_P1 times 4 dd 1 << (DESCALE_P1 - 1)
PD_DESCALE_P2 times 4 dd 1 << (DESCALE_P2 - 1)
PB_CENTERJSAMP times 16 db CENTERJSAMPLE
ALIGNZ 32
; --------------------------------------------------------------------------
SECTION SEG_TEXT
BITS 64
; Perform dequantization and inverse DCT on one block of coefficients.
;
; GLOBAL(void)
; jsimd_idct_islow_sse2(void *dct_table, JCOEFPTR coef_block,
; JSAMPARRAY output_buf, JDIMENSION output_col)
;
; r10 = jpeg_component_info *compptr
; r11 = JCOEFPTR coef_block
; r12 = JSAMPARRAY output_buf
; r13d = JDIMENSION output_col
%define wk(i) r15 - (WK_NUM - (i)) * SIZEOF_XMMWORD
; xmmword wk[WK_NUM]
%define WK_NUM 12
align 32
GLOBAL_FUNCTION(jsimd_idct_islow_sse2)
EXTN(jsimd_idct_islow_sse2):
ENDBR64
push rbp
mov rbp, rsp
push r15
and rsp, byte (-SIZEOF_XMMWORD) ; align to 128 bits
; Allocate stack space for wk array. r15 is used to access it.
mov r15, rsp
sub rsp, (SIZEOF_XMMWORD * WK_NUM)
COLLECT_ARGS 4
; ---- Pass 1: process columns from input.
mov rdx, r10 ; quantptr
mov rsi, r11 ; inptr
%ifndef NO_ZERO_COLUMN_TEST_ISLOW_SSE2
mov eax, dword [DWBLOCK(1, 0, rsi, SIZEOF_JCOEF)]
or eax, dword [DWBLOCK(2, 0, rsi, SIZEOF_JCOEF)]
jnz near .columnDCT
movdqa xmm0, XMMWORD [XMMBLOCK(1, 0, rsi, SIZEOF_JCOEF)]
movdqa xmm1, XMMWORD [XMMBLOCK(2, 0, rsi, SIZEOF_JCOEF)]
por xmm0, XMMWORD [XMMBLOCK(3, 0, rsi, SIZEOF_JCOEF)]
por xmm1, XMMWORD [XMMBLOCK(4, 0, rsi, SIZEOF_JCOEF)]
por xmm0, XMMWORD [XMMBLOCK(5, 0, rsi, SIZEOF_JCOEF)]
por xmm1, XMMWORD [XMMBLOCK(6, 0, rsi, SIZEOF_JCOEF)]
por xmm0, XMMWORD [XMMBLOCK(7, 0, rsi, SIZEOF_JCOEF)]
por xmm1, xmm0
packsswb xmm1, xmm1
packsswb xmm1, xmm1
movd eax, xmm1
test rax, rax
jnz short .columnDCT
; -- AC terms all zero
movdqa xmm5, XMMWORD [XMMBLOCK(0, 0, rsi, SIZEOF_JCOEF)]
pmullw xmm5, XMMWORD [XMMBLOCK(0, 0, rdx, SIZEOF_ISLOW_MULT_TYPE)]
psllw xmm5, PASS1_BITS
movdqa xmm4, xmm5 ; xmm5 = in0 = (00 01 02 03 04 05 06 07)
punpcklwd xmm5, xmm5 ; xmm5 = (00 00 01 01 02 02 03 03)
punpckhwd xmm4, xmm4 ; xmm4 = (04 04 05 05 06 06 07 07)
pshufd xmm7, xmm5, 0x00 ; xmm7 = col0 = (00 00 00 00 00 00 00 00)
pshufd xmm6, xmm5, 0x55 ; xmm6 = col1 = (01 01 01 01 01 01 01 01)
pshufd xmm1, xmm5, 0xAA ; xmm1 = col2 = (02 02 02 02 02 02 02 02)
pshufd xmm5, xmm5, 0xFF ; xmm5 = col3 = (03 03 03 03 03 03 03 03)
pshufd xmm0, xmm4, 0x00 ; xmm0 = col4 = (04 04 04 04 04 04 04 04)
pshufd xmm3, xmm4, 0x55 ; xmm3 = col5 = (05 05 05 05 05 05 05 05)
pshufd xmm2, xmm4, 0xAA ; xmm2 = col6 = (06 06 06 06 06 06 06 06)
pshufd xmm4, xmm4, 0xFF ; xmm4 = col7 = (07 07 07 07 07 07 07 07)
movdqa XMMWORD [wk(8)], xmm6 ; wk(8) = col1
movdqa XMMWORD [wk(9)], xmm5 ; wk(9) = col3
movdqa XMMWORD [wk(10)], xmm3 ; wk(10) = col5
movdqa XMMWORD [wk(11)], xmm4 ; wk(11) = col7
jmp near .column_end
%endif
.columnDCT:
; -- Even part
movdqa xmm0, XMMWORD [XMMBLOCK(0, 0, rsi, SIZEOF_JCOEF)]
movdqa xmm1, XMMWORD [XMMBLOCK(2, 0, rsi, SIZEOF_JCOEF)]
pmullw xmm0, XMMWORD [XMMBLOCK(0, 0, rdx, SIZEOF_ISLOW_MULT_TYPE)]
pmullw xmm1, XMMWORD [XMMBLOCK(2, 0, rdx, SIZEOF_ISLOW_MULT_TYPE)]
movdqa xmm2, XMMWORD [XMMBLOCK(4, 0, rsi, SIZEOF_JCOEF)]
movdqa xmm3, XMMWORD [XMMBLOCK(6, 0, rsi, SIZEOF_JCOEF)]
pmullw xmm2, XMMWORD [XMMBLOCK(4, 0, rdx, SIZEOF_ISLOW_MULT_TYPE)]
pmullw xmm3, XMMWORD [XMMBLOCK(6, 0, rdx, SIZEOF_ISLOW_MULT_TYPE)]
; (Original)
; z1 = (z2 + z3) * 0.541196100;
; tmp2 = z1 + z3 * -1.847759065;
; tmp3 = z1 + z2 * 0.765366865;
;
; (This implementation)
; tmp2 = z2 * 0.541196100 + z3 * (0.541196100 - 1.847759065);
; tmp3 = z2 * (0.541196100 + 0.765366865) + z3 * 0.541196100;
movdqa xmm4, xmm1 ; xmm1 = in2 = z2
movdqa xmm5, xmm1
punpcklwd xmm4, xmm3 ; xmm3 = in6 = z3
punpckhwd xmm5, xmm3
movdqa xmm1, xmm4
movdqa xmm3, xmm5
pmaddwd xmm4, [rel PW_F130_F054] ; xmm4 = tmp3L
pmaddwd xmm5, [rel PW_F130_F054] ; xmm5 = tmp3H
pmaddwd xmm1, [rel PW_F054_MF130] ; xmm1 = tmp2L
pmaddwd xmm3, [rel PW_F054_MF130] ; xmm3 = tmp2H
movdqa xmm6, xmm0
paddw xmm0, xmm2 ; xmm0 = in0 + in4
psubw xmm6, xmm2 ; xmm6 = in0 - in4
pxor xmm7, xmm7
pxor xmm2, xmm2
punpcklwd xmm7, xmm0 ; xmm7 = tmp0L
punpckhwd xmm2, xmm0 ; xmm2 = tmp0H
psrad xmm7, (16 - CONST_BITS)
; psrad xmm7, 16 & pslld xmm7, CONST_BITS
psrad xmm2, (16 - CONST_BITS)
; psrad xmm2, 16 & pslld xmm2, CONST_BITS
movdqa xmm0, xmm7
paddd xmm7, xmm4 ; xmm7 = tmp10L
psubd xmm0, xmm4 ; xmm0 = tmp13L
movdqa xmm4, xmm2
paddd xmm2, xmm5 ; xmm2 = tmp10H
psubd xmm4, xmm5 ; xmm4 = tmp13H
movdqa XMMWORD [wk(0)], xmm7 ; wk(0) = tmp10L
movdqa XMMWORD [wk(1)], xmm2 ; wk(1) = tmp10H
movdqa XMMWORD [wk(2)], xmm0 ; wk(2) = tmp13L
movdqa XMMWORD [wk(3)], xmm4 ; wk(3) = tmp13H
pxor xmm5, xmm5
pxor xmm7, xmm7
punpcklwd xmm5, xmm6 ; xmm5 = tmp1L
punpckhwd xmm7, xmm6 ; xmm7 = tmp1H
psrad xmm5, (16 - CONST_BITS)
; psrad xmm5, 16 & pslld xmm5, CONST_BITS
psrad xmm7, (16 - CONST_BITS)
; psrad xmm7, 16 & pslld xmm7, CONST_BITS
movdqa xmm2, xmm5
paddd xmm5, xmm1 ; xmm5 = tmp11L
psubd xmm2, xmm1 ; xmm2 = tmp12L
movdqa xmm0, xmm7
paddd xmm7, xmm3 ; xmm7 = tmp11H
psubd xmm0, xmm3 ; xmm0 = tmp12H
movdqa XMMWORD [wk(4)], xmm5 ; wk(4) = tmp11L
movdqa XMMWORD [wk(5)], xmm7 ; wk(5) = tmp11H
movdqa XMMWORD [wk(6)], xmm2 ; wk(6) = tmp12L
movdqa XMMWORD [wk(7)], xmm0 ; wk(7) = tmp12H
; -- Odd part
movdqa xmm4, XMMWORD [XMMBLOCK(1, 0, rsi, SIZEOF_JCOEF)]
movdqa xmm6, XMMWORD [XMMBLOCK(3, 0, rsi, SIZEOF_JCOEF)]
pmullw xmm4, XMMWORD [XMMBLOCK(1, 0, rdx, SIZEOF_ISLOW_MULT_TYPE)]
pmullw xmm6, XMMWORD [XMMBLOCK(3, 0, rdx, SIZEOF_ISLOW_MULT_TYPE)]
movdqa xmm1, XMMWORD [XMMBLOCK(5, 0, rsi, SIZEOF_JCOEF)]
movdqa xmm3, XMMWORD [XMMBLOCK(7, 0, rsi, SIZEOF_JCOEF)]
pmullw xmm1, XMMWORD [XMMBLOCK(5, 0, rdx, SIZEOF_ISLOW_MULT_TYPE)]
pmullw xmm3, XMMWORD [XMMBLOCK(7, 0, rdx, SIZEOF_ISLOW_MULT_TYPE)]
movdqa xmm5, xmm6
movdqa xmm7, xmm4
paddw xmm5, xmm3 ; xmm5 = z3
paddw xmm7, xmm1 ; xmm7 = z4
; (Original)
; z5 = (z3 + z4) * 1.175875602;
; z3 = z3 * -1.961570560; z4 = z4 * -0.390180644;
; z3 += z5; z4 += z5;
;
; (This implementation)
; z3 = z3 * (1.175875602 - 1.961570560) + z4 * 1.175875602;
; z4 = z3 * 1.175875602 + z4 * (1.175875602 - 0.390180644);
movdqa xmm2, xmm5
movdqa xmm0, xmm5
punpcklwd xmm2, xmm7
punpckhwd xmm0, xmm7
movdqa xmm5, xmm2
movdqa xmm7, xmm0
pmaddwd xmm2, [rel PW_MF078_F117] ; xmm2 = z3L
pmaddwd xmm0, [rel PW_MF078_F117] ; xmm0 = z3H
pmaddwd xmm5, [rel PW_F117_F078] ; xmm5 = z4L
pmaddwd xmm7, [rel PW_F117_F078] ; xmm7 = z4H
movdqa XMMWORD [wk(10)], xmm2 ; wk(10) = z3L
movdqa XMMWORD [wk(11)], xmm0 ; wk(11) = z3H
; (Original)
; z1 = tmp0 + tmp3; z2 = tmp1 + tmp2;
; tmp0 = tmp0 * 0.298631336; tmp1 = tmp1 * 2.053119869;
; tmp2 = tmp2 * 3.072711026; tmp3 = tmp3 * 1.501321110;
; z1 = z1 * -0.899976223; z2 = z2 * -2.562915447;
; tmp0 += z1 + z3; tmp1 += z2 + z4;
; tmp2 += z2 + z3; tmp3 += z1 + z4;
;
; (This implementation)
; tmp0 = tmp0 * (0.298631336 - 0.899976223) + tmp3 * -0.899976223;
; tmp1 = tmp1 * (2.053119869 - 2.562915447) + tmp2 * -2.562915447;
; tmp2 = tmp1 * -2.562915447 + tmp2 * (3.072711026 - 2.562915447);
; tmp3 = tmp0 * -0.899976223 + tmp3 * (1.501321110 - 0.899976223);
; tmp0 += z3; tmp1 += z4;
; tmp2 += z3; tmp3 += z4;
movdqa xmm2, xmm3
movdqa xmm0, xmm3
punpcklwd xmm2, xmm4
punpckhwd xmm0, xmm4
movdqa xmm3, xmm2
movdqa xmm4, xmm0
pmaddwd xmm2, [rel PW_MF060_MF089] ; xmm2 = tmp0L
pmaddwd xmm0, [rel PW_MF060_MF089] ; xmm0 = tmp0H
pmaddwd xmm3, [rel PW_MF089_F060] ; xmm3 = tmp3L
pmaddwd xmm4, [rel PW_MF089_F060] ; xmm4 = tmp3H
paddd xmm2, XMMWORD [wk(10)] ; xmm2 = tmp0L
paddd xmm0, XMMWORD [wk(11)] ; xmm0 = tmp0H
paddd xmm3, xmm5 ; xmm3 = tmp3L
paddd xmm4, xmm7 ; xmm4 = tmp3H
movdqa XMMWORD [wk(8)], xmm2 ; wk(8) = tmp0L
movdqa XMMWORD [wk(9)], xmm0 ; wk(9) = tmp0H
movdqa xmm2, xmm1
movdqa xmm0, xmm1
punpcklwd xmm2, xmm6
punpckhwd xmm0, xmm6
movdqa xmm1, xmm2
movdqa xmm6, xmm0
pmaddwd xmm2, [rel PW_MF050_MF256] ; xmm2 = tmp1L
pmaddwd xmm0, [rel PW_MF050_MF256] ; xmm0 = tmp1H
pmaddwd xmm1, [rel PW_MF256_F050] ; xmm1 = tmp2L
pmaddwd xmm6, [rel PW_MF256_F050] ; xmm6 = tmp2H
paddd xmm2, xmm5 ; xmm2 = tmp1L
paddd xmm0, xmm7 ; xmm0 = tmp1H
paddd xmm1, XMMWORD [wk(10)] ; xmm1 = tmp2L
paddd xmm6, XMMWORD [wk(11)] ; xmm6 = tmp2H
movdqa XMMWORD [wk(10)], xmm2 ; wk(10) = tmp1L
movdqa XMMWORD [wk(11)], xmm0 ; wk(11) = tmp1H
; -- Final output stage
movdqa xmm5, XMMWORD [wk(0)] ; xmm5 = tmp10L
movdqa xmm7, XMMWORD [wk(1)] ; xmm7 = tmp10H
movdqa xmm2, xmm5
movdqa xmm0, xmm7
paddd xmm5, xmm3 ; xmm5 = data0L
paddd xmm7, xmm4 ; xmm7 = data0H
psubd xmm2, xmm3 ; xmm2 = data7L
psubd xmm0, xmm4 ; xmm0 = data7H
movdqa xmm3, [rel PD_DESCALE_P1] ; xmm3 = [rel PD_DESCALE_P1]
paddd xmm5, xmm3
paddd xmm7, xmm3
psrad xmm5, DESCALE_P1
psrad xmm7, DESCALE_P1
paddd xmm2, xmm3
paddd xmm0, xmm3
psrad xmm2, DESCALE_P1
psrad xmm0, DESCALE_P1
packssdw xmm5, xmm7 ; xmm5 = data0 = (00 01 02 03 04 05 06 07)
packssdw xmm2, xmm0 ; xmm2 = data7 = (70 71 72 73 74 75 76 77)
movdqa xmm4, XMMWORD [wk(4)] ; xmm4 = tmp11L
movdqa xmm3, XMMWORD [wk(5)] ; xmm3 = tmp11H
movdqa xmm7, xmm4
movdqa xmm0, xmm3
paddd xmm4, xmm1 ; xmm4 = data1L
paddd xmm3, xmm6 ; xmm3 = data1H
psubd xmm7, xmm1 ; xmm7 = data6L
psubd xmm0, xmm6 ; xmm0 = data6H
movdqa xmm1, [rel PD_DESCALE_P1] ; xmm1 = [rel PD_DESCALE_P1]
paddd xmm4, xmm1
paddd xmm3, xmm1
psrad xmm4, DESCALE_P1
psrad xmm3, DESCALE_P1
paddd xmm7, xmm1
paddd xmm0, xmm1
psrad xmm7, DESCALE_P1
psrad xmm0, DESCALE_P1
packssdw xmm4, xmm3 ; xmm4 = data1 = (10 11 12 13 14 15 16 17)
packssdw xmm7, xmm0 ; xmm7 = data6 = (60 61 62 63 64 65 66 67)
movdqa xmm6, xmm5 ; transpose coefficients(phase 1)
punpcklwd xmm5, xmm4 ; xmm5 = (00 10 01 11 02 12 03 13)
punpckhwd xmm6, xmm4 ; xmm6 = (04 14 05 15 06 16 07 17)
movdqa xmm1, xmm7 ; transpose coefficients(phase 1)
punpcklwd xmm7, xmm2 ; xmm7 = (60 70 61 71 62 72 63 73)
punpckhwd xmm1, xmm2 ; xmm1 = (64 74 65 75 66 76 67 77)
movdqa xmm3, XMMWORD [wk(6)] ; xmm3 = tmp12L
movdqa xmm0, XMMWORD [wk(7)] ; xmm0 = tmp12H
movdqa xmm4, XMMWORD [wk(10)] ; xmm4 = tmp1L
movdqa xmm2, XMMWORD [wk(11)] ; xmm2 = tmp1H
movdqa XMMWORD [wk(0)], xmm5 ; wk(0) = (00 10 01 11 02 12 03 13)
movdqa XMMWORD [wk(1)], xmm6 ; wk(1) = (04 14 05 15 06 16 07 17)
movdqa XMMWORD [wk(4)], xmm7 ; wk(4) = (60 70 61 71 62 72 63 73)
movdqa XMMWORD [wk(5)], xmm1 ; wk(5) = (64 74 65 75 66 76 67 77)
movdqa xmm5, xmm3
movdqa xmm6, xmm0
paddd xmm3, xmm4 ; xmm3 = data2L
paddd xmm0, xmm2 ; xmm0 = data2H
psubd xmm5, xmm4 ; xmm5 = data5L
psubd xmm6, xmm2 ; xmm6 = data5H
movdqa xmm7, [rel PD_DESCALE_P1] ; xmm7 = [rel PD_DESCALE_P1]
paddd xmm3, xmm7
paddd xmm0, xmm7
psrad xmm3, DESCALE_P1
psrad xmm0, DESCALE_P1
paddd xmm5, xmm7
paddd xmm6, xmm7
psrad xmm5, DESCALE_P1
psrad xmm6, DESCALE_P1
packssdw xmm3, xmm0 ; xmm3 = data2 = (20 21 22 23 24 25 26 27)
packssdw xmm5, xmm6 ; xmm5 = data5 = (50 51 52 53 54 55 56 57)
movdqa xmm1, XMMWORD [wk(2)] ; xmm1 = tmp13L
movdqa xmm4, XMMWORD [wk(3)] ; xmm4 = tmp13H
movdqa xmm2, XMMWORD [wk(8)] ; xmm2 = tmp0L
movdqa xmm7, XMMWORD [wk(9)] ; xmm7 = tmp0H
movdqa xmm0, xmm1
movdqa xmm6, xmm4
paddd xmm1, xmm2 ; xmm1 = data3L
paddd xmm4, xmm7 ; xmm4 = data3H
psubd xmm0, xmm2 ; xmm0 = data4L
psubd xmm6, xmm7 ; xmm6 = data4H
movdqa xmm2, [rel PD_DESCALE_P1] ; xmm2 = [rel PD_DESCALE_P1]
paddd xmm1, xmm2
paddd xmm4, xmm2
psrad xmm1, DESCALE_P1
psrad xmm4, DESCALE_P1
paddd xmm0, xmm2
paddd xmm6, xmm2
psrad xmm0, DESCALE_P1
psrad xmm6, DESCALE_P1
packssdw xmm1, xmm4 ; xmm1 = data3 = (30 31 32 33 34 35 36 37)
packssdw xmm0, xmm6 ; xmm0 = data4 = (40 41 42 43 44 45 46 47)
movdqa xmm7, XMMWORD [wk(0)] ; xmm7 = (00 10 01 11 02 12 03 13)
movdqa xmm2, XMMWORD [wk(1)] ; xmm2 = (04 14 05 15 06 16 07 17)
movdqa xmm4, xmm3 ; transpose coefficients(phase 1)
punpcklwd xmm3, xmm1 ; xmm3 = (20 30 21 31 22 32 23 33)
punpckhwd xmm4, xmm1 ; xmm4 = (24 34 25 35 26 36 27 37)
movdqa xmm6, xmm0 ; transpose coefficients(phase 1)
punpcklwd xmm0, xmm5 ; xmm0 = (40 50 41 51 42 52 43 53)
punpckhwd xmm6, xmm5 ; xmm6 = (44 54 45 55 46 56 47 57)
movdqa xmm1, xmm7 ; transpose coefficients(phase 2)
punpckldq xmm7, xmm3 ; xmm7 = (00 10 20 30 01 11 21 31)
punpckhdq xmm1, xmm3 ; xmm1 = (02 12 22 32 03 13 23 33)
movdqa xmm5, xmm2 ; transpose coefficients(phase 2)
punpckldq xmm2, xmm4 ; xmm2 = (04 14 24 34 05 15 25 35)
punpckhdq xmm5, xmm4 ; xmm5 = (06 16 26 36 07 17 27 37)
movdqa xmm3, XMMWORD [wk(4)] ; xmm3 = (60 70 61 71 62 72 63 73)
movdqa xmm4, XMMWORD [wk(5)] ; xmm4 = (64 74 65 75 66 76 67 77)
movdqa XMMWORD [wk(6)], xmm2 ; wk(6) = (04 14 24 34 05 15 25 35)
movdqa XMMWORD [wk(7)], xmm5 ; wk(7) = (06 16 26 36 07 17 27 37)
movdqa xmm2, xmm0 ; transpose coefficients(phase 2)
punpckldq xmm0, xmm3 ; xmm0 = (40 50 60 70 41 51 61 71)
punpckhdq xmm2, xmm3 ; xmm2 = (42 52 62 72 43 53 63 73)
movdqa xmm5, xmm6 ; transpose coefficients(phase 2)
punpckldq xmm6, xmm4 ; xmm6 = (44 54 64 74 45 55 65 75)
punpckhdq xmm5, xmm4 ; xmm5 = (46 56 66 76 47 57 67 77)
movdqa xmm3, xmm7 ; transpose coefficients(phase 3)
punpcklqdq xmm7, xmm0 ; xmm7 = col0 = (00 10 20 30 40 50 60 70)
punpckhqdq xmm3, xmm0 ; xmm3 = col1 = (01 11 21 31 41 51 61 71)
movdqa xmm4, xmm1 ; transpose coefficients(phase 3)
punpcklqdq xmm1, xmm2 ; xmm1 = col2 = (02 12 22 32 42 52 62 72)
punpckhqdq xmm4, xmm2 ; xmm4 = col3 = (03 13 23 33 43 53 63 73)
movdqa xmm0, XMMWORD [wk(6)] ; xmm0 = (04 14 24 34 05 15 25 35)
movdqa xmm2, XMMWORD [wk(7)] ; xmm2 = (06 16 26 36 07 17 27 37)
movdqa XMMWORD [wk(8)], xmm3 ; wk(8) = col1
movdqa XMMWORD [wk(9)], xmm4 ; wk(9) = col3
movdqa xmm3, xmm0 ; transpose coefficients(phase 3)
punpcklqdq xmm0, xmm6 ; xmm0 = col4 = (04 14 24 34 44 54 64 74)
punpckhqdq xmm3, xmm6 ; xmm3 = col5 = (05 15 25 35 45 55 65 75)
movdqa xmm4, xmm2 ; transpose coefficients(phase 3)
punpcklqdq xmm2, xmm5 ; xmm2 = col6 = (06 16 26 36 46 56 66 76)
punpckhqdq xmm4, xmm5 ; xmm4 = col7 = (07 17 27 37 47 57 67 77)
movdqa XMMWORD [wk(10)], xmm3 ; wk(10) = col5
movdqa XMMWORD [wk(11)], xmm4 ; wk(11) = col7
.column_end:
; -- Prefetch the next coefficient block
prefetchnta [rsi + DCTSIZE2 * SIZEOF_JCOEF + 0 * 32]
prefetchnta [rsi + DCTSIZE2 * SIZEOF_JCOEF + 1 * 32]
prefetchnta [rsi + DCTSIZE2 * SIZEOF_JCOEF + 2 * 32]
prefetchnta [rsi + DCTSIZE2 * SIZEOF_JCOEF + 3 * 32]
; ---- Pass 2: process rows from work array, store into output array.
mov rdi, r12 ; (JSAMPROW *)
mov eax, r13d
; -- Even part
; xmm7 = col0, xmm1 = col2, xmm0 = col4, xmm2 = col6
; (Original)
; z1 = (z2 + z3) * 0.541196100;
; tmp2 = z1 + z3 * -1.847759065;
; tmp3 = z1 + z2 * 0.765366865;
;
; (This implementation)
; tmp2 = z2 * 0.541196100 + z3 * (0.541196100 - 1.847759065);
; tmp3 = z2 * (0.541196100 + 0.765366865) + z3 * 0.541196100;
movdqa xmm6, xmm1 ; xmm1 = in2 = z2
movdqa xmm5, xmm1
punpcklwd xmm6, xmm2 ; xmm2 = in6 = z3
punpckhwd xmm5, xmm2
movdqa xmm1, xmm6
movdqa xmm2, xmm5
pmaddwd xmm6, [rel PW_F130_F054] ; xmm6 = tmp3L
pmaddwd xmm5, [rel PW_F130_F054] ; xmm5 = tmp3H
pmaddwd xmm1, [rel PW_F054_MF130] ; xmm1 = tmp2L
pmaddwd xmm2, [rel PW_F054_MF130] ; xmm2 = tmp2H
movdqa xmm3, xmm7
paddw xmm7, xmm0 ; xmm7 = in0 + in4
psubw xmm3, xmm0 ; xmm3 = in0 - in4
pxor xmm4, xmm4
pxor xmm0, xmm0
punpcklwd xmm4, xmm7 ; xmm4 = tmp0L
punpckhwd xmm0, xmm7 ; xmm0 = tmp0H
psrad xmm4, (16 - CONST_BITS)
; psrad xmm4, 16 & pslld xmm4, CONST_BITS
psrad xmm0, (16 - CONST_BITS)
; psrad xmm0, 16 & pslld xmm0, CONST_BITS
movdqa xmm7, xmm4
paddd xmm4, xmm6 ; xmm4 = tmp10L
psubd xmm7, xmm6 ; xmm7 = tmp13L
movdqa xmm6, xmm0
paddd xmm0, xmm5 ; xmm0 = tmp10H
psubd xmm6, xmm5 ; xmm6 = tmp13H
movdqa XMMWORD [wk(0)], xmm4 ; wk(0) = tmp10L
movdqa XMMWORD [wk(1)], xmm0 ; wk(1) = tmp10H
movdqa XMMWORD [wk(2)], xmm7 ; wk(2) = tmp13L
movdqa XMMWORD [wk(3)], xmm6 ; wk(3) = tmp13H
pxor xmm5, xmm5
pxor xmm4, xmm4
punpcklwd xmm5, xmm3 ; xmm5 = tmp1L
punpckhwd xmm4, xmm3 ; xmm4 = tmp1H
psrad xmm5, (16 - CONST_BITS)
; psrad xmm5, 16 & pslld xmm5, CONST_BITS
psrad xmm4, (16 - CONST_BITS)
; psrad xmm4, 16 & pslld xmm4, CONST_BITS
movdqa xmm0, xmm5
paddd xmm5, xmm1 ; xmm5 = tmp11L
psubd xmm0, xmm1 ; xmm0 = tmp12L
movdqa xmm7, xmm4
paddd xmm4, xmm2 ; xmm4 = tmp11H
psubd xmm7, xmm2 ; xmm7 = tmp12H
movdqa XMMWORD [wk(4)], xmm5 ; wk(4) = tmp11L
movdqa XMMWORD [wk(5)], xmm4 ; wk(5) = tmp11H
movdqa XMMWORD [wk(6)], xmm0 ; wk(6) = tmp12L
movdqa XMMWORD [wk(7)], xmm7 ; wk(7) = tmp12H
; -- Odd part
movdqa xmm6, XMMWORD [wk(9)] ; xmm6 = col3
movdqa xmm3, XMMWORD [wk(8)] ; xmm3 = col1
movdqa xmm1, XMMWORD [wk(11)] ; xmm1 = col7
movdqa xmm2, XMMWORD [wk(10)] ; xmm2 = col5
movdqa xmm5, xmm6
movdqa xmm4, xmm3
paddw xmm5, xmm1 ; xmm5 = z3
paddw xmm4, xmm2 ; xmm4 = z4
; (Original)
; z5 = (z3 + z4) * 1.175875602;
; z3 = z3 * -1.961570560; z4 = z4 * -0.390180644;
; z3 += z5; z4 += z5;
;
; (This implementation)
; z3 = z3 * (1.175875602 - 1.961570560) + z4 * 1.175875602;
; z4 = z3 * 1.175875602 + z4 * (1.175875602 - 0.390180644);
movdqa xmm0, xmm5
movdqa xmm7, xmm5
punpcklwd xmm0, xmm4
punpckhwd xmm7, xmm4
movdqa xmm5, xmm0
movdqa xmm4, xmm7
pmaddwd xmm0, [rel PW_MF078_F117] ; xmm0 = z3L
pmaddwd xmm7, [rel PW_MF078_F117] ; xmm7 = z3H
pmaddwd xmm5, [rel PW_F117_F078] ; xmm5 = z4L
pmaddwd xmm4, [rel PW_F117_F078] ; xmm4 = z4H
movdqa XMMWORD [wk(10)], xmm0 ; wk(10) = z3L
movdqa XMMWORD [wk(11)], xmm7 ; wk(11) = z3H
; (Original)
; z1 = tmp0 + tmp3; z2 = tmp1 + tmp2;
; tmp0 = tmp0 * 0.298631336; tmp1 = tmp1 * 2.053119869;
; tmp2 = tmp2 * 3.072711026; tmp3 = tmp3 * 1.501321110;
; z1 = z1 * -0.899976223; z2 = z2 * -2.562915447;
; tmp0 += z1 + z3; tmp1 += z2 + z4;
; tmp2 += z2 + z3; tmp3 += z1 + z4;
;
; (This implementation)
; tmp0 = tmp0 * (0.298631336 - 0.899976223) + tmp3 * -0.899976223;
; tmp1 = tmp1 * (2.053119869 - 2.562915447) + tmp2 * -2.562915447;
; tmp2 = tmp1 * -2.562915447 + tmp2 * (3.072711026 - 2.562915447);
; tmp3 = tmp0 * -0.899976223 + tmp3 * (1.501321110 - 0.899976223);
; tmp0 += z3; tmp1 += z4;
; tmp2 += z3; tmp3 += z4;
movdqa xmm0, xmm1
movdqa xmm7, xmm1
punpcklwd xmm0, xmm3
punpckhwd xmm7, xmm3
movdqa xmm1, xmm0
movdqa xmm3, xmm7
pmaddwd xmm0, [rel PW_MF060_MF089] ; xmm0 = tmp0L
pmaddwd xmm7, [rel PW_MF060_MF089] ; xmm7 = tmp0H
pmaddwd xmm1, [rel PW_MF089_F060] ; xmm1 = tmp3L
pmaddwd xmm3, [rel PW_MF089_F060] ; xmm3 = tmp3H
paddd xmm0, XMMWORD [wk(10)] ; xmm0 = tmp0L
paddd xmm7, XMMWORD [wk(11)] ; xmm7 = tmp0H
paddd xmm1, xmm5 ; xmm1 = tmp3L
paddd xmm3, xmm4 ; xmm3 = tmp3H
movdqa XMMWORD [wk(8)], xmm0 ; wk(8) = tmp0L
movdqa XMMWORD [wk(9)], xmm7 ; wk(9) = tmp0H
movdqa xmm0, xmm2
movdqa xmm7, xmm2
punpcklwd xmm0, xmm6
punpckhwd xmm7, xmm6
movdqa xmm2, xmm0
movdqa xmm6, xmm7
pmaddwd xmm0, [rel PW_MF050_MF256] ; xmm0 = tmp1L
pmaddwd xmm7, [rel PW_MF050_MF256] ; xmm7 = tmp1H
pmaddwd xmm2, [rel PW_MF256_F050] ; xmm2 = tmp2L
pmaddwd xmm6, [rel PW_MF256_F050] ; xmm6 = tmp2H
paddd xmm0, xmm5 ; xmm0 = tmp1L
paddd xmm7, xmm4 ; xmm7 = tmp1H
paddd xmm2, XMMWORD [wk(10)] ; xmm2 = tmp2L
paddd xmm6, XMMWORD [wk(11)] ; xmm6 = tmp2H
movdqa XMMWORD [wk(10)], xmm0 ; wk(10) = tmp1L
movdqa XMMWORD [wk(11)], xmm7 ; wk(11) = tmp1H
; -- Final output stage
movdqa xmm5, XMMWORD [wk(0)] ; xmm5 = tmp10L
movdqa xmm4, XMMWORD [wk(1)] ; xmm4 = tmp10H
movdqa xmm0, xmm5
movdqa xmm7, xmm4
paddd xmm5, xmm1 ; xmm5 = data0L
paddd xmm4, xmm3 ; xmm4 = data0H
psubd xmm0, xmm1 ; xmm0 = data7L
psubd xmm7, xmm3 ; xmm7 = data7H
movdqa xmm1, [rel PD_DESCALE_P2] ; xmm1 = [rel PD_DESCALE_P2]
paddd xmm5, xmm1
paddd xmm4, xmm1
psrad xmm5, DESCALE_P2
psrad xmm4, DESCALE_P2
paddd xmm0, xmm1
paddd xmm7, xmm1
psrad xmm0, DESCALE_P2
psrad xmm7, DESCALE_P2
packssdw xmm5, xmm4 ; xmm5 = data0 = (00 10 20 30 40 50 60 70)
packssdw xmm0, xmm7 ; xmm0 = data7 = (07 17 27 37 47 57 67 77)
movdqa xmm3, XMMWORD [wk(4)] ; xmm3 = tmp11L
movdqa xmm1, XMMWORD [wk(5)] ; xmm1 = tmp11H
movdqa xmm4, xmm3
movdqa xmm7, xmm1
paddd xmm3, xmm2 ; xmm3 = data1L
paddd xmm1, xmm6 ; xmm1 = data1H
psubd xmm4, xmm2 ; xmm4 = data6L
psubd xmm7, xmm6 ; xmm7 = data6H
movdqa xmm2, [rel PD_DESCALE_P2] ; xmm2 = [rel PD_DESCALE_P2]
paddd xmm3, xmm2
paddd xmm1, xmm2
psrad xmm3, DESCALE_P2
psrad xmm1, DESCALE_P2
paddd xmm4, xmm2
paddd xmm7, xmm2
psrad xmm4, DESCALE_P2
psrad xmm7, DESCALE_P2
packssdw xmm3, xmm1 ; xmm3 = data1 = (01 11 21 31 41 51 61 71)
packssdw xmm4, xmm7 ; xmm4 = data6 = (06 16 26 36 46 56 66 76)
packsswb xmm5, xmm4
; xmm5 = (00 10 20 30 40 50 60 70 06 16 26 36 46 56 66 76)
packsswb xmm3, xmm0
; xmm3 = (01 11 21 31 41 51 61 71 07 17 27 37 47 57 67 77)
movdqa xmm6, XMMWORD [wk(6)] ; xmm6 = tmp12L
movdqa xmm2, XMMWORD [wk(7)] ; xmm2 = tmp12H
movdqa xmm1, XMMWORD [wk(10)] ; xmm1 = tmp1L
movdqa xmm7, XMMWORD [wk(11)] ; xmm7 = tmp1H
movdqa XMMWORD [wk(0)], xmm5
; wk(0) = (00 10 20 30 40 50 60 70 06 16 26 36 46 56 66 76)
movdqa XMMWORD [wk(1)], xmm3
; wk(1) = (01 11 21 31 41 51 61 71 07 17 27 37 47 57 67 77)
movdqa xmm4, xmm6
movdqa xmm0, xmm2
paddd xmm6, xmm1 ; xmm6 = data2L
paddd xmm2, xmm7 ; xmm2 = data2H
psubd xmm4, xmm1 ; xmm4 = data5L
psubd xmm0, xmm7 ; xmm0 = data5H
movdqa xmm5, [rel PD_DESCALE_P2] ; xmm5 = [rel PD_DESCALE_P2]
paddd xmm6, xmm5
paddd xmm2, xmm5
psrad xmm6, DESCALE_P2
psrad xmm2, DESCALE_P2
paddd xmm4, xmm5
paddd xmm0, xmm5
psrad xmm4, DESCALE_P2
psrad xmm0, DESCALE_P2
packssdw xmm6, xmm2 ; xmm6 = data2 = (02 12 22 32 42 52 62 72)
packssdw xmm4, xmm0 ; xmm4 = data5 = (05 15 25 35 45 55 65 75)
movdqa xmm3, XMMWORD [wk(2)] ; xmm3 = tmp13L
movdqa xmm1, XMMWORD [wk(3)] ; xmm1 = tmp13H
movdqa xmm7, XMMWORD [wk(8)] ; xmm7 = tmp0L
movdqa xmm5, XMMWORD [wk(9)] ; xmm5 = tmp0H
movdqa xmm2, xmm3
movdqa xmm0, xmm1
paddd xmm3, xmm7 ; xmm3 = data3L
paddd xmm1, xmm5 ; xmm1 = data3H
psubd xmm2, xmm7 ; xmm2 = data4L
psubd xmm0, xmm5 ; xmm0 = data4H
movdqa xmm7, [rel PD_DESCALE_P2] ; xmm7 = [rel PD_DESCALE_P2]
paddd xmm3, xmm7
paddd xmm1, xmm7
psrad xmm3, DESCALE_P2
psrad xmm1, DESCALE_P2
paddd xmm2, xmm7
paddd xmm0, xmm7
psrad xmm2, DESCALE_P2
psrad xmm0, DESCALE_P2
movdqa xmm5, [rel PB_CENTERJSAMP] ; xmm5 = [rel PB_CENTERJSAMP]
packssdw xmm3, xmm1 ; xmm3 = data3 = (03 13 23 33 43 53 63 73)
packssdw xmm2, xmm0 ; xmm2 = data4 = (04 14 24 34 44 54 64 74)
movdqa xmm7, XMMWORD [wk(0)]
; xmm7 = (00 10 20 30 40 50 60 70 06 16 26 36 46 56 66 76)
movdqa xmm1, XMMWORD [wk(1)]
; xmm1 = (01 11 21 31 41 51 61 71 07 17 27 37 47 57 67 77)
packsswb xmm6, xmm2
; xmm6 = (02 12 22 32 42 52 62 72 04 14 24 34 44 54 64 74)
packsswb xmm3, xmm4
; xmm3 = (03 13 23 33 43 53 63 73 05 15 25 35 45 55 65 75)
paddb xmm7, xmm5
paddb xmm1, xmm5
paddb xmm6, xmm5
paddb xmm3, xmm5
; transpose coefficients(phase 1)
movdqa xmm0, xmm7
punpcklbw xmm7, xmm1
; xmm7 = (00 01 10 11 20 21 30 31 40 41 50 51 60 61 70 71)
punpckhbw xmm0, xmm1
; xmm0 = (06 07 16 17 26 27 36 37 46 47 56 57 66 67 76 77)
movdqa xmm2, xmm6
punpcklbw xmm6, xmm3
; xmm6 = (02 03 12 13 22 23 32 33 42 43 52 53 62 63 72 73)
punpckhbw xmm2, xmm3
; xmm2 = (04 05 14 15 24 25 34 35 44 45 54 55 64 65 74 75)
; transpose coefficients(phase 2)
movdqa xmm4, xmm7
punpcklwd xmm7, xmm6
; xmm7 = (00 01 02 03 10 11 12 13 20 21 22 23 30 31 32 33)
punpckhwd xmm4, xmm6
; xmm4 = (40 41 42 43 50 51 52 53 60 61 62 63 70 71 72 73)
movdqa xmm5, xmm2
punpcklwd xmm2, xmm0
; xmm2 = (04 05 06 07 14 15 16 17 24 25 26 27 34 35 36 37)
punpckhwd xmm5, xmm0
; xmm5 = (44 45 46 47 54 55 56 57 64 65 66 67 74 75 76 77)
; transpose coefficients(phase 3)
movdqa xmm1, xmm7
punpckldq xmm7, xmm2
; xmm7 = (00 01 02 03 04 05 06 07 10 11 12 13 14 15 16 17)
punpckhdq xmm1, xmm2
; xmm1 = (20 21 22 23 24 25 26 27 30 31 32 33 34 35 36 37)
movdqa xmm3, xmm4
punpckldq xmm4, xmm5
; xmm4 = (40 41 42 43 44 45 46 47 50 51 52 53 54 55 56 57)
punpckhdq xmm3, xmm5
; xmm3 = (60 61 62 63 64 65 66 67 70 71 72 73 74 75 76 77)
pshufd xmm6, xmm7, 0x4E
; xmm6 = (10 11 12 13 14 15 16 17 00 01 02 03 04 05 06 07)
pshufd xmm0, xmm1, 0x4E
; xmm0 = (30 31 32 33 34 35 36 37 20 21 22 23 24 25 26 27)
pshufd xmm2, xmm4, 0x4E
; xmm2 = (50 51 52 53 54 55 56 57 40 41 42 43 44 45 46 47)
pshufd xmm5, xmm3, 0x4E
; xmm5 = (70 71 72 73 74 75 76 77 60 61 62 63 64 65 66 67)
mov rdxp, JSAMPROW [rdi + 0 * SIZEOF_JSAMPROW]
mov rsip, JSAMPROW [rdi + 2 * SIZEOF_JSAMPROW]
movq XMM_MMWORD [rdx + rax * SIZEOF_JSAMPLE], xmm7
movq XMM_MMWORD [rsi + rax * SIZEOF_JSAMPLE], xmm1
mov rdxp, JSAMPROW [rdi + 4 * SIZEOF_JSAMPROW]
mov rsip, JSAMPROW [rdi + 6 * SIZEOF_JSAMPROW]
movq XMM_MMWORD [rdx + rax * SIZEOF_JSAMPLE], xmm4
movq XMM_MMWORD [rsi + rax * SIZEOF_JSAMPLE], xmm3
mov rdxp, JSAMPROW [rdi + 1 * SIZEOF_JSAMPROW]
mov rsip, JSAMPROW [rdi + 3 * SIZEOF_JSAMPROW]
movq XMM_MMWORD [rdx + rax * SIZEOF_JSAMPLE], xmm6
movq XMM_MMWORD [rsi + rax * SIZEOF_JSAMPLE], xmm0
mov rdxp, JSAMPROW [rdi + 5 * SIZEOF_JSAMPROW]
mov rsip, JSAMPROW [rdi + 7 * SIZEOF_JSAMPROW]
movq XMM_MMWORD [rdx + rax * SIZEOF_JSAMPLE], xmm2
movq XMM_MMWORD [rsi + rax * SIZEOF_JSAMPLE], xmm5
UNCOLLECT_ARGS 4
lea rsp, [rbp - 8]
pop r15
pop rbp
ret
; For some reason, the OS X linker does not honor the request to align the
; segment unless we do this.
align 32
lib/libjpeg-turbo/simd/x86_64/jidctred-sse2.asm
Executable
+577
View File
@@ -0,0 +1,577 @@
;
; Reduced-size IDCT (64-bit SSE2)
;
; Copyright 2009 Pierre Ossman <[email protected]> for Cendio AB
; Copyright (C) 2009, 2016, 2024-2025, D. R. Commander.
; Copyright (C) 2018, Matthias Räncker.
; Copyright (C) 2023, Aliaksiej Kandracienka.
;
; Based on the x86 SIMD extension for IJG JPEG library
; Copyright (C) 1999-2006, MIYASAKA Masaru.
; For conditions of distribution and use, see copyright notice in jsimdext.inc
;
; This file should be assembled with NASM (Netwide Assembler) or Yasm.
;
; This file contains inverse DCT routines that produce reduced-size output:
; either 4x4 or 2x2 pixels from an 8x8 DCT block. The following code is based
; directly on the IJG's original jidctred.c; see jidctred.c for more details.
%include "jsimdext.inc"
%include "jdct.inc"
; --------------------------------------------------------------------------
%define CONST_BITS 13
%define PASS1_BITS 2
%define DESCALE_P1_4 (CONST_BITS - PASS1_BITS + 1)
%define DESCALE_P2_4 (CONST_BITS + PASS1_BITS + 3 + 1)
%define DESCALE_P1_2 (CONST_BITS - PASS1_BITS + 2)
%define DESCALE_P2_2 (CONST_BITS + PASS1_BITS + 3 + 2)
%if CONST_BITS == 13
F_0_211 equ 1730 ; FIX(0.211164243)
F_0_509 equ 4176 ; FIX(0.509795579)
F_0_601 equ 4926 ; FIX(0.601344887)
F_0_720 equ 5906 ; FIX(0.720959822)
F_0_765 equ 6270 ; FIX(0.765366865)
F_0_850 equ 6967 ; FIX(0.850430095)
F_0_899 equ 7373 ; FIX(0.899976223)
F_1_061 equ 8697 ; FIX(1.061594337)
F_1_272 equ 10426 ; FIX(1.272758580)
F_1_451 equ 11893 ; FIX(1.451774981)
F_1_847 equ 15137 ; FIX(1.847759065)
F_2_172 equ 17799 ; FIX(2.172734803)
F_2_562 equ 20995 ; FIX(2.562915447)
F_3_624 equ 29692 ; FIX(3.624509785)
%else
; NASM cannot do compile-time arithmetic on floating-point constants.
%define DESCALE(x, n) (((x) + (1 << ((n) - 1))) >> (n))
F_0_211 equ DESCALE( 226735879, 30 - CONST_BITS) ; FIX(0.211164243)
F_0_509 equ DESCALE( 547388834, 30 - CONST_BITS) ; FIX(0.509795579)
F_0_601 equ DESCALE( 645689155, 30 - CONST_BITS) ; FIX(0.601344887)
F_0_720 equ DESCALE( 774124714, 30 - CONST_BITS) ; FIX(0.720959822)
F_0_765 equ DESCALE( 821806413, 30 - CONST_BITS) ; FIX(0.765366865)
F_0_850 equ DESCALE( 913142361, 30 - CONST_BITS) ; FIX(0.850430095)
F_0_899 equ DESCALE( 966342111, 30 - CONST_BITS) ; FIX(0.899976223)
F_1_061 equ DESCALE(1139878239, 30 - CONST_BITS) ; FIX(1.061594337)
F_1_272 equ DESCALE(1366614119, 30 - CONST_BITS) ; FIX(1.272758580)
F_1_451 equ DESCALE(1558831516, 30 - CONST_BITS) ; FIX(1.451774981)
F_1_847 equ DESCALE(1984016188, 30 - CONST_BITS) ; FIX(1.847759065)
F_2_172 equ DESCALE(2332956230, 30 - CONST_BITS) ; FIX(2.172734803)
F_2_562 equ DESCALE(2751909506, 30 - CONST_BITS) ; FIX(2.562915447)
F_3_624 equ DESCALE(3891787747, 30 - CONST_BITS) ; FIX(3.624509785)
%endif
; --------------------------------------------------------------------------
SECTION SEG_CONST
ALIGNZ 32
GLOBAL_DATA(jconst_idct_red_sse2)
EXTN(jconst_idct_red_sse2):
PW_F184_MF076 times 4 dw F_1_847, -F_0_765
PW_F256_F089 times 4 dw F_2_562, F_0_899
PW_F106_MF217 times 4 dw F_1_061, -F_2_172
PW_MF060_MF050 times 4 dw -F_0_601, -F_0_509
PW_F145_MF021 times 4 dw F_1_451, -F_0_211
PW_F362_MF127 times 4 dw F_3_624, -F_1_272
PW_F085_MF072 times 4 dw F_0_850, -F_0_720
PD_DESCALE_P1_4 times 4 dd 1 << (DESCALE_P1_4 - 1)
PD_DESCALE_P2_4 times 4 dd 1 << (DESCALE_P2_4 - 1)
PD_DESCALE_P1_2 times 4 dd 1 << (DESCALE_P1_2 - 1)
PD_DESCALE_P2_2 times 4 dd 1 << (DESCALE_P2_2 - 1)
PB_CENTERJSAMP times 16 db CENTERJSAMPLE
ALIGNZ 32
; --------------------------------------------------------------------------
SECTION SEG_TEXT
BITS 64
; Perform dequantization and inverse DCT on one block of coefficients,
; producing a reduced-size 4x4 output block.
;
; GLOBAL(void)
; jsimd_idct_4x4_sse2(void *dct_table, JCOEFPTR coef_block,
; JSAMPARRAY output_buf, JDIMENSION output_col)
;
; r10 = void *dct_table
; r11 = JCOEFPTR coef_block
; r12 = JSAMPARRAY output_buf
; r13d = JDIMENSION output_col
%define wk(i) r15 - (WK_NUM - (i)) * SIZEOF_XMMWORD ; xmmword wk[WK_NUM]
%define WK_NUM 2
align 32
GLOBAL_FUNCTION(jsimd_idct_4x4_sse2)
EXTN(jsimd_idct_4x4_sse2):
ENDBR64
push rbp
mov rbp, rsp
push r15
and rsp, byte (-SIZEOF_XMMWORD) ; align to 128 bits
; Allocate stack space for wk array. r15 is used to access it.
mov r15, rsp
sub rsp, byte (SIZEOF_XMMWORD * WK_NUM)
COLLECT_ARGS 4
; ---- Pass 1: process columns from input.
mov rdx, r10 ; quantptr
mov rsi, r11 ; inptr
%ifndef NO_ZERO_COLUMN_TEST_4X4_SSE2
mov eax, dword [DWBLOCK(1, 0, rsi, SIZEOF_JCOEF)]
or eax, dword [DWBLOCK(2, 0, rsi, SIZEOF_JCOEF)]
jnz short .columnDCT
movdqa xmm0, XMMWORD [XMMBLOCK(1, 0, rsi, SIZEOF_JCOEF)]
movdqa xmm1, XMMWORD [XMMBLOCK(2, 0, rsi, SIZEOF_JCOEF)]
por xmm0, XMMWORD [XMMBLOCK(3, 0, rsi, SIZEOF_JCOEF)]
por xmm1, XMMWORD [XMMBLOCK(5, 0, rsi, SIZEOF_JCOEF)]
por xmm0, XMMWORD [XMMBLOCK(6, 0, rsi, SIZEOF_JCOEF)]
por xmm1, XMMWORD [XMMBLOCK(7, 0, rsi, SIZEOF_JCOEF)]
por xmm0, xmm1
packsswb xmm0, xmm0
packsswb xmm0, xmm0
movd eax, xmm0
test rax, rax
jnz short .columnDCT
; -- AC terms all zero
movdqa xmm0, XMMWORD [XMMBLOCK(0, 0, rsi, SIZEOF_JCOEF)]
pmullw xmm0, XMMWORD [XMMBLOCK(0, 0, rdx, SIZEOF_ISLOW_MULT_TYPE)]
psllw xmm0, PASS1_BITS
movdqa xmm3, xmm0 ; xmm0 = in0 = (00 01 02 03 04 05 06 07)
punpcklwd xmm0, xmm0 ; xmm0 = (00 00 01 01 02 02 03 03)
punpckhwd xmm3, xmm3 ; xmm3 = (04 04 05 05 06 06 07 07)
pshufd xmm1, xmm0, 0x50
; xmm1 = [col0 col1] = (00 00 00 00 01 01 01 01)
pshufd xmm0, xmm0, 0xFA
; xmm0 = [col2 col3] = (02 02 02 02 03 03 03 03)
pshufd xmm6, xmm3, 0x50
; xmm6 = [col4 col5] = (04 04 04 04 05 05 05 05)
pshufd xmm3, xmm3, 0xFA
; xmm3 = [col6 col7] = (06 06 06 06 07 07 07 07)
jmp near .column_end
%endif
.columnDCT:
; -- Odd part
movdqa xmm0, XMMWORD [XMMBLOCK(1, 0, rsi, SIZEOF_JCOEF)]
movdqa xmm1, XMMWORD [XMMBLOCK(3, 0, rsi, SIZEOF_JCOEF)]
pmullw xmm0, XMMWORD [XMMBLOCK(1, 0, rdx, SIZEOF_ISLOW_MULT_TYPE)]
pmullw xmm1, XMMWORD [XMMBLOCK(3, 0, rdx, SIZEOF_ISLOW_MULT_TYPE)]
movdqa xmm2, XMMWORD [XMMBLOCK(5, 0, rsi, SIZEOF_JCOEF)]
movdqa xmm3, XMMWORD [XMMBLOCK(7, 0, rsi, SIZEOF_JCOEF)]
pmullw xmm2, XMMWORD [XMMBLOCK(5, 0, rdx, SIZEOF_ISLOW_MULT_TYPE)]
pmullw xmm3, XMMWORD [XMMBLOCK(7, 0, rdx, SIZEOF_ISLOW_MULT_TYPE)]
movdqa xmm4, xmm0
movdqa xmm5, xmm0
punpcklwd xmm4, xmm1
punpckhwd xmm5, xmm1
movdqa xmm0, xmm4
movdqa xmm1, xmm5
pmaddwd xmm4, [rel PW_F256_F089] ; xmm4 = (tmp2L)
pmaddwd xmm5, [rel PW_F256_F089] ; xmm5 = (tmp2H)
pmaddwd xmm0, [rel PW_F106_MF217] ; xmm0 = (tmp0L)
pmaddwd xmm1, [rel PW_F106_MF217] ; xmm1 = (tmp0H)
movdqa xmm6, xmm2
movdqa xmm7, xmm2
punpcklwd xmm6, xmm3
punpckhwd xmm7, xmm3
movdqa xmm2, xmm6
movdqa xmm3, xmm7
pmaddwd xmm6, [rel PW_MF060_MF050] ; xmm6 = (tmp2L)
pmaddwd xmm7, [rel PW_MF060_MF050] ; xmm7 = (tmp2H)
pmaddwd xmm2, [rel PW_F145_MF021] ; xmm2 = (tmp0L)
pmaddwd xmm3, [rel PW_F145_MF021] ; xmm3 = (tmp0H)
paddd xmm6, xmm4 ; xmm6 = tmp2L
paddd xmm7, xmm5 ; xmm7 = tmp2H
paddd xmm2, xmm0 ; xmm2 = tmp0L
paddd xmm3, xmm1 ; xmm3 = tmp0H
movdqa XMMWORD [wk(0)], xmm2 ; wk(0) = tmp0L
movdqa XMMWORD [wk(1)], xmm3 ; wk(1) = tmp0H
; -- Even part
movdqa xmm4, XMMWORD [XMMBLOCK(0, 0, rsi, SIZEOF_JCOEF)]
movdqa xmm5, XMMWORD [XMMBLOCK(2, 0, rsi, SIZEOF_JCOEF)]
movdqa xmm0, XMMWORD [XMMBLOCK(6, 0, rsi, SIZEOF_JCOEF)]
pmullw xmm4, XMMWORD [XMMBLOCK(0, 0, rdx, SIZEOF_ISLOW_MULT_TYPE)]
pmullw xmm5, XMMWORD [XMMBLOCK(2, 0, rdx, SIZEOF_ISLOW_MULT_TYPE)]
pmullw xmm0, XMMWORD [XMMBLOCK(6, 0, rdx, SIZEOF_ISLOW_MULT_TYPE)]
pxor xmm1, xmm1
pxor xmm2, xmm2
punpcklwd xmm1, xmm4 ; xmm1 = tmp0L
punpckhwd xmm2, xmm4 ; xmm2 = tmp0H
psrad xmm1, (16 - CONST_BITS - 1)
; psrad xmm1, 16 & pslld xmm1, CONST_BITS + 1
psrad xmm2, (16 - CONST_BITS - 1)
; psrad xmm2, 16 & pslld xmm2, CONST_BITS + 1
movdqa xmm3, xmm5 ; xmm5 = in2 = z2
punpcklwd xmm5, xmm0 ; xmm0 = in6 = z3
punpckhwd xmm3, xmm0
pmaddwd xmm5, [rel PW_F184_MF076] ; xmm5 = tmp2L
pmaddwd xmm3, [rel PW_F184_MF076] ; xmm3 = tmp2H
movdqa xmm4, xmm1
movdqa xmm0, xmm2
paddd xmm1, xmm5 ; xmm1 = tmp10L
paddd xmm2, xmm3 ; xmm2 = tmp10H
psubd xmm4, xmm5 ; xmm4 = tmp12L
psubd xmm0, xmm3 ; xmm0 = tmp12H
; -- Final output stage
movdqa xmm5, xmm1
movdqa xmm3, xmm2
paddd xmm1, xmm6 ; xmm1 = data0L
paddd xmm2, xmm7 ; xmm2 = data0H
psubd xmm5, xmm6 ; xmm5 = data3L
psubd xmm3, xmm7 ; xmm3 = data3H
movdqa xmm6, [rel PD_DESCALE_P1_4] ; xmm6 = [rel PD_DESCALE_P1_4]
paddd xmm1, xmm6
paddd xmm2, xmm6
psrad xmm1, DESCALE_P1_4
psrad xmm2, DESCALE_P1_4
paddd xmm5, xmm6
paddd xmm3, xmm6
psrad xmm5, DESCALE_P1_4
psrad xmm3, DESCALE_P1_4
packssdw xmm1, xmm2 ; xmm1 = data0 = (00 01 02 03 04 05 06 07)
packssdw xmm5, xmm3 ; xmm5 = data3 = (30 31 32 33 34 35 36 37)
movdqa xmm7, XMMWORD [wk(0)] ; xmm7 = tmp0L
movdqa xmm6, XMMWORD [wk(1)] ; xmm6 = tmp0H
movdqa xmm2, xmm4
movdqa xmm3, xmm0
paddd xmm4, xmm7 ; xmm4 = data1L
paddd xmm0, xmm6 ; xmm0 = data1H
psubd xmm2, xmm7 ; xmm2 = data2L
psubd xmm3, xmm6 ; xmm3 = data2H
movdqa xmm7, [rel PD_DESCALE_P1_4] ; xmm7 = [rel PD_DESCALE_P1_4]
paddd xmm4, xmm7
paddd xmm0, xmm7
psrad xmm4, DESCALE_P1_4
psrad xmm0, DESCALE_P1_4
paddd xmm2, xmm7
paddd xmm3, xmm7
psrad xmm2, DESCALE_P1_4
psrad xmm3, DESCALE_P1_4
packssdw xmm4, xmm0 ; xmm4 = data1 = (10 11 12 13 14 15 16 17)
packssdw xmm2, xmm3 ; xmm2 = data2 = (20 21 22 23 24 25 26 27)
movdqa xmm6, xmm1 ; transpose coefficients(phase 1)
punpcklwd xmm1, xmm4 ; xmm1 = (00 10 01 11 02 12 03 13)
punpckhwd xmm6, xmm4 ; xmm6 = (04 14 05 15 06 16 07 17)
movdqa xmm7, xmm2 ; transpose coefficients(phase 1)
punpcklwd xmm2, xmm5 ; xmm2 = (20 30 21 31 22 32 23 33)
punpckhwd xmm7, xmm5 ; xmm7 = (24 34 25 35 26 36 27 37)
movdqa xmm0, xmm1 ; transpose coefficients(phase 2)
punpckldq xmm1, xmm2 ; xmm1 = [col0 col1] = (00 10 20 30 01 11 21 31)
punpckhdq xmm0, xmm2 ; xmm0 = [col2 col3] = (02 12 22 32 03 13 23 33)
movdqa xmm3, xmm6 ; transpose coefficients(phase 2)
punpckldq xmm6, xmm7 ; xmm6 = [col4 col5] = (04 14 24 34 05 15 25 35)
punpckhdq xmm3, xmm7 ; xmm3 = [col6 col7] = (06 16 26 36 07 17 27 37)
.column_end:
; -- Prefetch the next coefficient block
prefetchnta [rsi + DCTSIZE2 * SIZEOF_JCOEF + 0 * 32]
prefetchnta [rsi + DCTSIZE2 * SIZEOF_JCOEF + 1 * 32]
prefetchnta [rsi + DCTSIZE2 * SIZEOF_JCOEF + 2 * 32]
prefetchnta [rsi + DCTSIZE2 * SIZEOF_JCOEF + 3 * 32]
; ---- Pass 2: process rows, store into output array.
mov rdi, r12 ; (JSAMPROW *)
mov eax, r13d
; -- Even part
pxor xmm4, xmm4
punpcklwd xmm4, xmm1 ; xmm4 = tmp0
psrad xmm4, (16 - CONST_BITS - 1)
; psrad xmm4, 16 & pslld xmm4, CONST_BITS + 1
; -- Odd part
punpckhwd xmm1, xmm0
punpckhwd xmm6, xmm3
movdqa xmm5, xmm1
movdqa xmm2, xmm6
pmaddwd xmm1, [rel PW_F256_F089] ; xmm1 = (tmp2)
pmaddwd xmm6, [rel PW_MF060_MF050] ; xmm6 = (tmp2)
pmaddwd xmm5, [rel PW_F106_MF217] ; xmm5 = (tmp0)
pmaddwd xmm2, [rel PW_F145_MF021] ; xmm2 = (tmp0)
paddd xmm6, xmm1 ; xmm6 = tmp2
paddd xmm2, xmm5 ; xmm2 = tmp0
; -- Even part
punpcklwd xmm0, xmm3
pmaddwd xmm0, [rel PW_F184_MF076] ; xmm0 = tmp2
movdqa xmm7, xmm4
paddd xmm4, xmm0 ; xmm4 = tmp10
psubd xmm7, xmm0 ; xmm7 = tmp12
; -- Final output stage
movdqa xmm1, [rel PD_DESCALE_P2_4] ; xmm1 = [rel PD_DESCALE_P2_4]
movdqa xmm5, xmm4
movdqa xmm3, xmm7
paddd xmm4, xmm6 ; xmm4 = data0 = (00 10 20 30)
paddd xmm7, xmm2 ; xmm7 = data1 = (01 11 21 31)
psubd xmm5, xmm6 ; xmm5 = data3 = (03 13 23 33)
psubd xmm3, xmm2 ; xmm3 = data2 = (02 12 22 32)
paddd xmm4, xmm1
paddd xmm7, xmm1
psrad xmm4, DESCALE_P2_4
psrad xmm7, DESCALE_P2_4
paddd xmm5, xmm1
paddd xmm3, xmm1
psrad xmm5, DESCALE_P2_4
psrad xmm3, DESCALE_P2_4
packssdw xmm4, xmm3 ; xmm4 = (00 10 20 30 02 12 22 32)
packssdw xmm7, xmm5 ; xmm7 = (01 11 21 31 03 13 23 33)
movdqa xmm0, xmm4 ; transpose coefficients(phase 1)
punpcklwd xmm4, xmm7 ; xmm4 = (00 01 10 11 20 21 30 31)
punpckhwd xmm0, xmm7 ; xmm0 = (02 03 12 13 22 23 32 33)
movdqa xmm6, xmm4 ; transpose coefficients(phase 2)
punpckldq xmm4, xmm0 ; xmm4 = (00 01 02 03 10 11 12 13)
punpckhdq xmm6, xmm0 ; xmm6 = (20 21 22 23 30 31 32 33)
packsswb xmm4, xmm6 ; xmm4 = (00 01 02 03 10 11 12 13 20 ..)
paddb xmm4, [rel PB_CENTERJSAMP]
pshufd xmm2, xmm4, 0x39 ; xmm2 = (10 11 12 13 20 21 22 23 30 ..)
pshufd xmm1, xmm4, 0x4E ; xmm1 = (20 21 22 23 30 31 32 33 00 ..)
pshufd xmm3, xmm4, 0x93 ; xmm3 = (30 31 32 33 00 01 02 03 10 ..)
mov rdxp, JSAMPROW [rdi + 0 * SIZEOF_JSAMPROW]
mov rsip, JSAMPROW [rdi + 1 * SIZEOF_JSAMPROW]
movd XMM_DWORD [rdx + rax * SIZEOF_JSAMPLE], xmm4
movd XMM_DWORD [rsi + rax * SIZEOF_JSAMPLE], xmm2
mov rdxp, JSAMPROW [rdi + 2 * SIZEOF_JSAMPROW]
mov rsip, JSAMPROW [rdi + 3 * SIZEOF_JSAMPROW]
movd XMM_DWORD [rdx + rax * SIZEOF_JSAMPLE], xmm1
movd XMM_DWORD [rsi + rax * SIZEOF_JSAMPLE], xmm3
UNCOLLECT_ARGS 4
lea rsp, [rbp - 8]
pop r15
pop rbp
ret
; --------------------------------------------------------------------------
;
; Perform dequantization and inverse DCT on one block of coefficients,
; producing a reduced-size 2x2 output block.
;
; GLOBAL(void)
; jsimd_idct_2x2_sse2(void *dct_table, JCOEFPTR coef_block,
; JSAMPARRAY output_buf, JDIMENSION output_col)
;
; r10 = void *dct_table
; r11 = JCOEFPTR coef_block
; r12 = JSAMPARRAY output_buf
; r13d = JDIMENSION output_col
align 32
GLOBAL_FUNCTION(jsimd_idct_2x2_sse2)
EXTN(jsimd_idct_2x2_sse2):
ENDBR64
push rbp
mov rbp, rsp
COLLECT_ARGS 4
push rbx
; ---- Pass 1: process columns from input.
mov rdx, r10 ; quantptr
mov rsi, r11 ; inptr
; | input: | result: |
; | 00 01 ** 03 ** 05 ** 07 | |
; | 10 11 ** 13 ** 15 ** 17 | |
; | ** ** ** ** ** ** ** ** | |
; | 30 31 ** 33 ** 35 ** 37 | A0 A1 A3 A5 A7 |
; | ** ** ** ** ** ** ** ** | B0 B1 B3 B5 B7 |
; | 50 51 ** 53 ** 55 ** 57 | |
; | ** ** ** ** ** ** ** ** | |
; | 70 71 ** 73 ** 75 ** 77 | |
; -- Odd part
movdqa xmm0, XMMWORD [XMMBLOCK(1, 0, rsi, SIZEOF_JCOEF)]
movdqa xmm1, XMMWORD [XMMBLOCK(3, 0, rsi, SIZEOF_JCOEF)]
pmullw xmm0, XMMWORD [XMMBLOCK(1, 0, rdx, SIZEOF_ISLOW_MULT_TYPE)]
pmullw xmm1, XMMWORD [XMMBLOCK(3, 0, rdx, SIZEOF_ISLOW_MULT_TYPE)]
movdqa xmm2, XMMWORD [XMMBLOCK(5, 0, rsi, SIZEOF_JCOEF)]
movdqa xmm3, XMMWORD [XMMBLOCK(7, 0, rsi, SIZEOF_JCOEF)]
pmullw xmm2, XMMWORD [XMMBLOCK(5, 0, rdx, SIZEOF_ISLOW_MULT_TYPE)]
pmullw xmm3, XMMWORD [XMMBLOCK(7, 0, rdx, SIZEOF_ISLOW_MULT_TYPE)]
; xmm0 = (10 11 ** 13 ** 15 ** 17)
; xmm1 = (30 31 ** 33 ** 35 ** 37)
; xmm2 = (50 51 ** 53 ** 55 ** 57)
; xmm3 = (70 71 ** 73 ** 75 ** 77)
pcmpeqd xmm7, xmm7
pslld xmm7, WORD_BIT ; xmm7 = { 0x0000 0xFFFF 0x0000 0xFFFF .. }
movdqa xmm4, xmm0 ; xmm4 = (10 11 ** 13 ** 15 ** 17)
movdqa xmm5, xmm2 ; xmm5 = (50 51 ** 53 ** 55 ** 57)
punpcklwd xmm4, xmm1 ; xmm4 = (10 30 11 31 ** ** 13 33)
punpcklwd xmm5, xmm3 ; xmm5 = (50 70 51 71 ** ** 53 73)
pmaddwd xmm4, [rel PW_F362_MF127]
pmaddwd xmm5, [rel PW_F085_MF072]
psrld xmm0, WORD_BIT ; xmm0 = (11 -- 13 -- 15 -- 17 --)
pand xmm1, xmm7 ; xmm1 = (-- 31 -- 33 -- 35 -- 37)
psrld xmm2, WORD_BIT ; xmm2 = (51 -- 53 -- 55 -- 57 --)
pand xmm3, xmm7 ; xmm3 = (-- 71 -- 73 -- 75 -- 77)
por xmm0, xmm1 ; xmm0 = (11 31 13 33 15 35 17 37)
por xmm2, xmm3 ; xmm2 = (51 71 53 73 55 75 57 77)
pmaddwd xmm0, [rel PW_F362_MF127]
pmaddwd xmm2, [rel PW_F085_MF072]
paddd xmm4, xmm5 ; xmm4 = tmp0[col0 col1 **** col3]
paddd xmm0, xmm2 ; xmm0 = tmp0[col1 col3 col5 col7]
; -- Even part
movdqa xmm6, XMMWORD [XMMBLOCK(0, 0, rsi, SIZEOF_JCOEF)]
pmullw xmm6, XMMWORD [XMMBLOCK(0, 0, rdx, SIZEOF_ISLOW_MULT_TYPE)]
; xmm6 = (00 01 ** 03 ** 05 ** 07)
movdqa xmm1, xmm6 ; xmm1 = (00 01 ** 03 ** 05 ** 07)
pslld xmm6, WORD_BIT ; xmm6 = (-- 00 -- ** -- ** -- **)
pand xmm1, xmm7 ; xmm1 = (-- 01 -- 03 -- 05 -- 07)
psrad xmm6, (WORD_BIT - CONST_BITS - 2)
; xmm6 = tmp10[col0 **** **** ****]
psrad xmm1, (WORD_BIT - CONST_BITS - 2)
; xmm1 = tmp10[col1 col3 col5 col7]
; -- Final output stage
movdqa xmm3, xmm6
movdqa xmm5, xmm1
paddd xmm6, xmm4 ; xmm6 = data0[col0 **** **** ****] = (A0 ** ** **)
paddd xmm1, xmm0 ; xmm1 = data0[col1 col3 col5 col7] = (A1 A3 A5 A7)
psubd xmm3, xmm4 ; xmm3 = data1[col0 **** **** ****] = (B0 ** ** **)
psubd xmm5, xmm0 ; xmm5 = data1[col1 col3 col5 col7] = (B1 B3 B5 B7)
movdqa xmm2, [rel PD_DESCALE_P1_2] ; xmm2 = [rel PD_DESCALE_P1_2]
punpckldq xmm6, xmm3 ; xmm6 = (A0 B0 ** **)
movdqa xmm7, xmm1
punpcklqdq xmm1, xmm5 ; xmm1 = (A1 A3 B1 B3)
punpckhqdq xmm7, xmm5 ; xmm7 = (A5 A7 B5 B7)
paddd xmm6, xmm2
psrad xmm6, DESCALE_P1_2
paddd xmm1, xmm2
paddd xmm7, xmm2
psrad xmm1, DESCALE_P1_2
psrad xmm7, DESCALE_P1_2
; -- Prefetch the next coefficient block
prefetchnta [rsi + DCTSIZE2 * SIZEOF_JCOEF + 0 * 32]
prefetchnta [rsi + DCTSIZE2 * SIZEOF_JCOEF + 1 * 32]
prefetchnta [rsi + DCTSIZE2 * SIZEOF_JCOEF + 2 * 32]
prefetchnta [rsi + DCTSIZE2 * SIZEOF_JCOEF + 3 * 32]
; ---- Pass 2: process rows, store into output array.
mov rdi, r12 ; (JSAMPROW *)
mov eax, r13d
; | input:| result:|
; | A0 B0 | |
; | A1 B1 | C0 C1 |
; | A3 B3 | D0 D1 |
; | A5 B5 | |
; | A7 B7 | |
; -- Odd part
packssdw xmm1, xmm1 ; xmm1 = (A1 A3 B1 B3 A1 A3 B1 B3)
packssdw xmm7, xmm7 ; xmm7 = (A5 A7 B5 B7 A5 A7 B5 B7)
pmaddwd xmm1, [rel PW_F362_MF127]
pmaddwd xmm7, [rel PW_F085_MF072]
paddd xmm1, xmm7 ; xmm1 = tmp0[row0 row1 row0 row1]
; -- Even part
pslld xmm6, (CONST_BITS + 2) ; xmm6 = tmp10[row0 row1 **** ****]
; -- Final output stage
movdqa xmm4, xmm6
paddd xmm6, xmm1 ; xmm6 = data0[row0 row1 **** ****] = (C0 C1 ** **)
psubd xmm4, xmm1 ; xmm4 = data1[row0 row1 **** ****] = (D0 D1 ** **)
punpckldq xmm6, xmm4 ; xmm6 = (C0 D0 C1 D1)
paddd xmm6, [rel PD_DESCALE_P2_2]
psrad xmm6, DESCALE_P2_2
packssdw xmm6, xmm6 ; xmm6 = (C0 D0 C1 D1 C0 D0 C1 D1)
packsswb xmm6, xmm6 ; xmm6 = (C0 D0 C1 D1 C0 D0 C1 D1 ..)
paddb xmm6, [rel PB_CENTERJSAMP]
pextrw ebx, xmm6, 0x00 ; ebx = (C0 D0 -- --)
pextrw ecx, xmm6, 0x01 ; ecx = (C1 D1 -- --)
mov rdxp, JSAMPROW [rdi + 0 * SIZEOF_JSAMPROW]
mov rsip, JSAMPROW [rdi + 1 * SIZEOF_JSAMPROW]
mov word [rdx + rax * SIZEOF_JSAMPLE], bx
mov word [rsi + rax * SIZEOF_JSAMPLE], cx
pop rbx
UNCOLLECT_ARGS 4
pop rbp
ret
; For some reason, the OS X linker does not honor the request to align the
; segment unless we do this.
align 32
lib/libjpeg-turbo/simd/x86_64/jquantf-sse2.asm
Executable
+149
View File
@@ -0,0 +1,149 @@
;
; Sample data conversion and quantization (64-bit SSE & SSE2)
;
; Copyright 2009 Pierre Ossman <[email protected]> for Cendio AB
; Copyright (C) 2009, 2016, 2024-2025, D. R. Commander.
; Copyright (C) 2018, Matthias Räncker.
;
; Based on the x86 SIMD extension for IJG JPEG library
; Copyright (C) 1999-2006, MIYASAKA Masaru.
; For conditions of distribution and use, see copyright notice in jsimdext.inc
;
; This file should be assembled with NASM (Netwide Assembler) or Yasm.
%include "jsimdext.inc"
%include "jdct.inc"
; --------------------------------------------------------------------------
SECTION SEG_TEXT
BITS 64
; Load data into workspace, applying unsigned->signed conversion
;
; GLOBAL(void)
; jsimd_convsamp_float_sse2(JSAMPARRAY sample_data, JDIMENSION start_col,
; FAST_FLOAT *workspace)
;
; r10 = JSAMPARRAY sample_data
; r11d = JDIMENSION start_col
; r12 = FAST_FLOAT *workspace
align 32
GLOBAL_FUNCTION(jsimd_convsamp_float_sse2)
EXTN(jsimd_convsamp_float_sse2):
ENDBR64
push rbp
mov rbp, rsp
COLLECT_ARGS 3
push rbx
pcmpeqw xmm7, xmm7
psllw xmm7, 7
packsswb xmm7, xmm7 ; xmm7 = PB_CENTERJSAMPLE (0x808080..)
mov rsi, r10
mov eax, r11d
mov rdi, r12
mov rcx, DCTSIZE / 2
.convloop:
mov rbxp, JSAMPROW [rsi + 0 * SIZEOF_JSAMPROW] ; (JSAMPLE *)
mov rdxp, JSAMPROW [rsi + 1 * SIZEOF_JSAMPROW] ; (JSAMPLE *)
movq xmm0, XMM_MMWORD [rbx + rax * SIZEOF_JSAMPLE]
movq xmm1, XMM_MMWORD [rdx + rax * SIZEOF_JSAMPLE]
psubb xmm0, xmm7 ; xmm0 = (01234567)
psubb xmm1, xmm7 ; xmm1 = (89abcdef)
punpcklbw xmm0, xmm0 ; xmm0 = (*0*1*2*3*4*5*6*7)
punpcklbw xmm1, xmm1 ; xmm1 = (*8*9*a*b*c*d*e*f)
punpcklwd xmm2, xmm0 ; xmm2 = (***0***1***2***3)
punpckhwd xmm0, xmm0 ; xmm0 = (***4***5***6***7)
punpcklwd xmm3, xmm1 ; xmm3 = (***8***9***a***b)
punpckhwd xmm1, xmm1 ; xmm1 = (***c***d***e***f)
psrad xmm2, (DWORD_BIT - BYTE_BIT) ; xmm2 = (0123)
psrad xmm0, (DWORD_BIT - BYTE_BIT) ; xmm0 = (4567)
cvtdq2ps xmm2, xmm2 ; xmm2 = (0123)
cvtdq2ps xmm0, xmm0 ; xmm0 = (4567)
psrad xmm3, (DWORD_BIT - BYTE_BIT) ; xmm3 = (89ab)
psrad xmm1, (DWORD_BIT - BYTE_BIT) ; xmm1 = (cdef)
cvtdq2ps xmm3, xmm3 ; xmm3 = (89ab)
cvtdq2ps xmm1, xmm1 ; xmm1 = (cdef)
movaps XMMWORD [XMMBLOCK(0, 0, rdi, SIZEOF_FAST_FLOAT)], xmm2
movaps XMMWORD [XMMBLOCK(0, 1, rdi, SIZEOF_FAST_FLOAT)], xmm0
movaps XMMWORD [XMMBLOCK(1, 0, rdi, SIZEOF_FAST_FLOAT)], xmm3
movaps XMMWORD [XMMBLOCK(1, 1, rdi, SIZEOF_FAST_FLOAT)], xmm1
add rsi, byte 2 * SIZEOF_JSAMPROW
add rdi, byte 2 * DCTSIZE * SIZEOF_FAST_FLOAT
dec rcx
jnz short .convloop
pop rbx
UNCOLLECT_ARGS 3
pop rbp
ret
; --------------------------------------------------------------------------
;
; Quantize/descale the coefficients, and store into coef_block
;
; GLOBAL(void)
; jsimd_quantize_float_sse2(JCOEFPTR coef_block, FAST_FLOAT *divisors,
; FAST_FLOAT *workspace)
;
; r10 = JCOEFPTR coef_block
; r11 = FAST_FLOAT *divisors
; r12 = FAST_FLOAT *workspace
align 32
GLOBAL_FUNCTION(jsimd_quantize_float_sse2)
EXTN(jsimd_quantize_float_sse2):
ENDBR64
push rbp
mov rbp, rsp
COLLECT_ARGS 3
mov rsi, r12
mov rdx, r11
mov rdi, r10
mov rax, DCTSIZE2 / 16
.quantloop:
movaps xmm0, XMMWORD [XMMBLOCK(0, 0, rsi, SIZEOF_FAST_FLOAT)]
movaps xmm1, XMMWORD [XMMBLOCK(0, 1, rsi, SIZEOF_FAST_FLOAT)]
mulps xmm0, XMMWORD [XMMBLOCK(0, 0, rdx, SIZEOF_FAST_FLOAT)]
mulps xmm1, XMMWORD [XMMBLOCK(0, 1, rdx, SIZEOF_FAST_FLOAT)]
movaps xmm2, XMMWORD [XMMBLOCK(1, 0, rsi, SIZEOF_FAST_FLOAT)]
movaps xmm3, XMMWORD [XMMBLOCK(1, 1, rsi, SIZEOF_FAST_FLOAT)]
mulps xmm2, XMMWORD [XMMBLOCK(1, 0, rdx, SIZEOF_FAST_FLOAT)]
mulps xmm3, XMMWORD [XMMBLOCK(1, 1, rdx, SIZEOF_FAST_FLOAT)]
cvtps2dq xmm0, xmm0
cvtps2dq xmm1, xmm1
cvtps2dq xmm2, xmm2
cvtps2dq xmm3, xmm3
packssdw xmm0, xmm1
packssdw xmm2, xmm3
movdqa XMMWORD [XMMBLOCK(0, 0, rdi, SIZEOF_JCOEF)], xmm0
movdqa XMMWORD [XMMBLOCK(1, 0, rdi, SIZEOF_JCOEF)], xmm2
add rsi, byte 16 * SIZEOF_FAST_FLOAT
add rdx, byte 16 * SIZEOF_FAST_FLOAT
add rdi, byte 16 * SIZEOF_JCOEF
dec rax
jnz short .quantloop
UNCOLLECT_ARGS 3
pop rbp
ret
; For some reason, the OS X linker does not honor the request to align the
; segment unless we do this.
align 32
lib/libjpeg-turbo/simd/x86_64/jquanti-sse2.asm
Executable
+187
View File
@@ -0,0 +1,187 @@
;
; Sample data conversion and quantization (64-bit SSE2)
;
; Copyright 2009 Pierre Ossman <[email protected]> for Cendio AB
; Copyright (C) 2009, 2016, 2024-2025, D. R. Commander.
; Copyright (C) 2018, Matthias Räncker.
;
; Based on the x86 SIMD extension for IJG JPEG library
; Copyright (C) 1999-2006, MIYASAKA Masaru.
; For conditions of distribution and use, see copyright notice in jsimdext.inc
;
; This file should be assembled with NASM (Netwide Assembler) or Yasm.
%include "jsimdext.inc"
%include "jdct.inc"
; --------------------------------------------------------------------------
SECTION SEG_TEXT
BITS 64
; Load data into workspace, applying unsigned->signed conversion
;
; GLOBAL(void)
; jsimd_convsamp_sse2(JSAMPARRAY sample_data, JDIMENSION start_col,
; DCTELEM *workspace)
;
; r10 = JSAMPARRAY sample_data
; r11d = JDIMENSION start_col
; r12 = DCTELEM *workspace
align 32
GLOBAL_FUNCTION(jsimd_convsamp_sse2)
EXTN(jsimd_convsamp_sse2):
ENDBR64
push rbp
mov rbp, rsp
COLLECT_ARGS 3
push rbx
pxor xmm6, xmm6 ; xmm6 = (all 0's)
pcmpeqw xmm7, xmm7
psllw xmm7, 7 ; xmm7 = { 0xFF80 0xFF80 0xFF80 0xFF80 .. }
mov rsi, r10
mov eax, r11d
mov rdi, r12
mov rcx, DCTSIZE / 4
.convloop:
mov rbxp, JSAMPROW [rsi + 0 * SIZEOF_JSAMPROW] ; (JSAMPLE *)
mov rdxp, JSAMPROW [rsi + 1 * SIZEOF_JSAMPROW] ; (JSAMPLE *)
movq xmm0, XMM_MMWORD [rbx + rax * SIZEOF_JSAMPLE]
; xmm0 = (01234567)
movq xmm1, XMM_MMWORD [rdx + rax * SIZEOF_JSAMPLE]
; xmm1 = (89abcdef)
mov rbxp, JSAMPROW [rsi + 2 * SIZEOF_JSAMPROW] ; (JSAMPLE *)
mov rdxp, JSAMPROW [rsi + 3 * SIZEOF_JSAMPROW] ; (JSAMPLE *)
movq xmm2, XMM_MMWORD [rbx + rax * SIZEOF_JSAMPLE]
; xmm2 = (ghijklmn)
movq xmm3, XMM_MMWORD [rdx + rax * SIZEOF_JSAMPLE]
; xmm3 = (opqrstuv)
punpcklbw xmm0, xmm6 ; xmm0 = (01234567)
punpcklbw xmm1, xmm6 ; xmm1 = (89abcdef)
paddw xmm0, xmm7
paddw xmm1, xmm7
punpcklbw xmm2, xmm6 ; xmm2 = (ghijklmn)
punpcklbw xmm3, xmm6 ; xmm3 = (opqrstuv)
paddw xmm2, xmm7
paddw xmm3, xmm7
movdqa XMMWORD [XMMBLOCK(0, 0, rdi, SIZEOF_DCTELEM)], xmm0
movdqa XMMWORD [XMMBLOCK(1, 0, rdi, SIZEOF_DCTELEM)], xmm1
movdqa XMMWORD [XMMBLOCK(2, 0, rdi, SIZEOF_DCTELEM)], xmm2
movdqa XMMWORD [XMMBLOCK(3, 0, rdi, SIZEOF_DCTELEM)], xmm3
add rsi, byte 4 * SIZEOF_JSAMPROW
add rdi, byte 4 * DCTSIZE * SIZEOF_DCTELEM
dec rcx
jnz short .convloop
pop rbx
UNCOLLECT_ARGS 3
pop rbp
ret
; --------------------------------------------------------------------------
;
; Quantize/descale the coefficients, and store into coef_block
;
; This implementation is based on an algorithm described in
; "Optimizing subroutines in assembly language:
; An optimization guide for x86 platforms" (https://agner.org/optimize).
;
; GLOBAL(void)
; jsimd_quantize_sse2(JCOEFPTR coef_block, DCTELEM *divisors,
; DCTELEM *workspace)
%define RECIPROCAL(m, n, b) \
XMMBLOCK(DCTSIZE * 0 + (m), (n), (b), SIZEOF_DCTELEM)
%define CORRECTION(m, n, b) \
XMMBLOCK(DCTSIZE * 1 + (m), (n), (b), SIZEOF_DCTELEM)
%define SCALE(m, n, b) \
XMMBLOCK(DCTSIZE * 2 + (m), (n), (b), SIZEOF_DCTELEM)
; r10 = JCOEFPTR coef_block
; r11 = DCTELEM *divisors
; r12 = DCTELEM *workspace
align 32
GLOBAL_FUNCTION(jsimd_quantize_sse2)
EXTN(jsimd_quantize_sse2):
ENDBR64
push rbp
mov rbp, rsp
COLLECT_ARGS 3
mov rsi, r12
mov rdx, r11
mov rdi, r10
mov rax, DCTSIZE2 / 32
.quantloop:
movdqa xmm4, XMMWORD [XMMBLOCK(0, 0, rsi, SIZEOF_DCTELEM)]
movdqa xmm5, XMMWORD [XMMBLOCK(1, 0, rsi, SIZEOF_DCTELEM)]
movdqa xmm6, XMMWORD [XMMBLOCK(2, 0, rsi, SIZEOF_DCTELEM)]
movdqa xmm7, XMMWORD [XMMBLOCK(3, 0, rsi, SIZEOF_DCTELEM)]
movdqa xmm0, xmm4
movdqa xmm1, xmm5
movdqa xmm2, xmm6
movdqa xmm3, xmm7
psraw xmm4, (WORD_BIT - 1)
psraw xmm5, (WORD_BIT - 1)
psraw xmm6, (WORD_BIT - 1)
psraw xmm7, (WORD_BIT - 1)
pxor xmm0, xmm4
pxor xmm1, xmm5
pxor xmm2, xmm6
pxor xmm3, xmm7
psubw xmm0, xmm4 ; if (xmm0 < 0) xmm0 = -xmm0;
psubw xmm1, xmm5 ; if (xmm1 < 0) xmm1 = -xmm1;
psubw xmm2, xmm6 ; if (xmm2 < 0) xmm2 = -xmm2;
psubw xmm3, xmm7 ; if (xmm3 < 0) xmm3 = -xmm3;
paddw xmm0, XMMWORD [CORRECTION(0, 0, rdx)]
; correction + roundfactor
paddw xmm1, XMMWORD [CORRECTION(1, 0, rdx)]
paddw xmm2, XMMWORD [CORRECTION(2, 0, rdx)]
paddw xmm3, XMMWORD [CORRECTION(3, 0, rdx)]
pmulhuw xmm0, XMMWORD [RECIPROCAL(0, 0, rdx)] ; reciprocal
pmulhuw xmm1, XMMWORD [RECIPROCAL(1, 0, rdx)]
pmulhuw xmm2, XMMWORD [RECIPROCAL(2, 0, rdx)]
pmulhuw xmm3, XMMWORD [RECIPROCAL(3, 0, rdx)]
pmulhuw xmm0, XMMWORD [SCALE(0, 0, rdx)] ; scale
pmulhuw xmm1, XMMWORD [SCALE(1, 0, rdx)]
pmulhuw xmm2, XMMWORD [SCALE(2, 0, rdx)]
pmulhuw xmm3, XMMWORD [SCALE(3, 0, rdx)]
pxor xmm0, xmm4
pxor xmm1, xmm5
pxor xmm2, xmm6
pxor xmm3, xmm7
psubw xmm0, xmm4
psubw xmm1, xmm5
psubw xmm2, xmm6
psubw xmm3, xmm7
movdqa XMMWORD [XMMBLOCK(0, 0, rdi, SIZEOF_DCTELEM)], xmm0
movdqa XMMWORD [XMMBLOCK(1, 0, rdi, SIZEOF_DCTELEM)], xmm1
movdqa XMMWORD [XMMBLOCK(2, 0, rdi, SIZEOF_DCTELEM)], xmm2
movdqa XMMWORD [XMMBLOCK(3, 0, rdi, SIZEOF_DCTELEM)], xmm3
add rsi, byte 32 * SIZEOF_DCTELEM
add rdx, byte 32 * SIZEOF_DCTELEM
add rdi, byte 32 * SIZEOF_JCOEF
dec rax
jnz near .quantloop
UNCOLLECT_ARGS 3
pop rbp
ret
; For some reason, the OS X linker does not honor the request to align the
; segment unless we do this.
align 32
lib/libjpeg-turbo/simd/x86_64/jsimd.c
Executable
+1108
View File
File diff suppressed because it is too large Load Diff
lib/libjpeg-turbo/test/croptest.in
Executable
+95
View File
@@ -0,0 +1,95 @@
#!/bin/bash
set -u
set -e
trap onexit INT
trap onexit TERM
trap onexit EXIT
onexit()
{
if [ -d $OUTDIR ]; then
rm -rf $OUTDIR
fi
}
runme()
{
echo \*\*\* $*
$*
}
IMAGE=vgl_6548_0026a.bmp
WIDTH=128
HEIGHT=95
IMGDIR=@CMAKE_SOURCE_DIR@/testimages
OUTDIR=`mktemp -d /tmp/__croptest_output.XXXXXX`
EXEDIR=@CMAKE_BINARY_DIR@
if [ -d $OUTDIR ]; then
rm -rf $OUTDIR
fi
mkdir -p $OUTDIR
exec >$EXEDIR/test/croptest.log
echo "============================================================"
echo "$IMAGE ($WIDTH x $HEIGHT)"
echo "============================================================"
echo
for PROGARG in "" -progressive; do
cp $IMGDIR/$IMAGE $OUTDIR
basename=`basename $IMAGE .bmp`
echo "------------------------------------------------------------"
echo "Generating test images"
echo "------------------------------------------------------------"
echo
runme $EXEDIR/cjpeg $PROGARG -grayscale -outfile $OUTDIR/${basename}_GRAY.jpg $IMGDIR/${basename}.bmp
runme $EXEDIR/cjpeg $PROGARG -sample 2x2 -outfile $OUTDIR/${basename}_420.jpg $IMGDIR/${basename}.bmp
runme $EXEDIR/cjpeg $PROGARG -sample 2x1 -outfile $OUTDIR/${basename}_422.jpg $IMGDIR/${basename}.bmp
runme $EXEDIR/cjpeg $PROGARG -sample 1x2 -outfile $OUTDIR/${basename}_440.jpg $IMGDIR/${basename}.bmp
runme $EXEDIR/cjpeg $PROGARG -sample 1x1 -outfile $OUTDIR/${basename}_444.jpg $IMGDIR/${basename}.bmp
echo
for NSARG in "" -nosmooth; do
for COLORSARG in "" "-colors 256 -dither none -onepass"; do
for Y in {0..16}; do
for H in {1..16}; do
X=$(( (Y*16)%128 ))
W=$(( WIDTH-X-7 ))
if [ $Y -le 15 ]; then
CROPSPEC="${W}x${H}+${X}+${Y}"
else
Y2=$(( HEIGHT-H ));
CROPSPEC="${W}x${H}+${X}+${Y2}"
fi
echo "------------------------------------------------------------"
echo $PROGARG $NSARG $COLORSARG -crop $CROPSPEC
echo "------------------------------------------------------------"
echo
for samp in GRAY 420 422 440 444; do
$EXEDIR/djpeg $NSARG $COLORSARG -rgb -outfile $OUTDIR/${basename}_${samp}_full.ppm $OUTDIR/${basename}_${samp}.jpg
magick $OUTDIR/${basename}_${samp}_full.ppm -crop $CROPSPEC $OUTDIR/${basename}_${samp}_ref.ppm
runme $EXEDIR/djpeg $NSARG $COLORSARG -crop $CROPSPEC -rgb -outfile $OUTDIR/${basename}_${samp}.ppm $OUTDIR/${basename}_${samp}.jpg
runme cmp $OUTDIR/${basename}_${samp}.ppm $OUTDIR/${basename}_${samp}_ref.ppm
done
echo
done
done
done
done
done
echo SUCCESS!
lib/libjpeg-turbo/test/tjbenchtest.in
Executable
+453
View File
@@ -0,0 +1,453 @@
#!/bin/bash
set -u
set -e
trap onexit INT
trap onexit TERM
trap onexit EXIT
onexit()
{
if [ -d $OUTDIR ]; then
rm -rf $OUTDIR
fi
}
runme()
{
echo \*\*\* $*
"$@"
}
EXT=bmp
IMAGES="vgl_5674_0098.${EXT} vgl_6434_0018a.${EXT} vgl_6548_0026a.${EXT} shira_bird8.${EXT}"
IMGDIR=@CMAKE_SOURCE_DIR@/testimages
OUTDIR=`mktemp -d /tmp/__tjbenchtest_output.XXXXXX`
EXEDIR=@CMAKE_BINARY_DIR@
JAVA="@Java_JAVA_EXECUTABLE@"
JAVAARGS="-cp $EXEDIR/java/turbojpeg.jar -Djava.library.path=$EXEDIR"
TJBENCH=$EXEDIR/tjbench
BMPARG=
NSARG=
YUVARG=
ALLOC=0
ALLOCARG=
ENTROPYARG=
JAVAARG=
LOSSLSARG=
LOSSLSPSV=
TJQUAL=95
x1SUBSAMP="444 GRAY"
x24SUBSAMP="422 440 420 411 441"
ALLSUBSAMP="444 422 440 420 411 441 GRAY"
PRECISION=8
if [ "$EXT" = "bmp" ]; then BMPARG=-b; fi
if [ -d $OUTDIR ]; then
rm -rf $OUTDIR
fi
mkdir -p $OUTDIR
while [ $# -gt 0 ]; do
case "$1" in
-yuv)
NSARG=-nos
YUVARG=-yuv
# NOTE: The combination of tj3EncodeYUV*() and tj3CompressFromYUV*() does not
# always produce bitwise-identical results to tj3Compress*() if subsampling is
# enabled. In both cases, if the image width or height are not evenly
# divisible by the MCU width/height, then the bottom and/or right edge are
# expanded. However, the libjpeg code performs this expansion prior to
# downsampling, and TurboJPEG performs it in tj3CompressFromYUV*(), which is
# after downsampling. Thus, the two will agree only if the width/height along
# each downsampled dimension is an odd number or is evenly divisible by the MCU
# width/height. This disagreement basically amounts to a round-off error, but
# there is no easy way around it, so for now, we just test the only image that
# works. (NOTE: shira_bird8 does not suffer from the above issue, but it
# suffers from an unrelated problem whereby the combination of
# tj3DecompressToYUV*() and tj3DecodeYUV*() does not produce bitwise-identical
# results to tj3Decompress*() if decompression scaling is enabled. This latter
# phenomenon is not yet fully understood but is also believed to be some sort
# of round-off error.)
IMAGES="vgl_6548_0026a.${EXT}"
;;
-alloc)
ALLOCARG=-alloc
ALLOC=1
;;
-java)
JAVAARG=-java
TJBENCH="$JAVA $JAVAARGS TJBench"
;;
-optimize)
ENTROPYARG=-optimize
;;
-progressive)
if [ "$ENTROPYARG" = "-arithmetic" ]; then
ENTROPYARG=-progressive-arithmetic
else
ENTROPYARG=-progressive
fi
;;
-arithmetic)
if [ "$ENTROPYARG" = "-progressive" ]; then
ENTROPYARG=-progressive-arithmetic
else
ENTROPYARG=-arithmetic
fi
;;
-lossless)
LOSSLSARG="-lossless"
LOSSLSPSV=4
TJQUAL=4
x1SUBSAMP=444
x24SUBSAMP=444
ALLSUBSAMP=444
;;
-precision)
shift
PRECISION=$1
if [ $PRECISION != 8 ]; then
EXT=ppm
IMAGES="monkey16.${EXT}"
BMPARG=
fi
;;
esac
shift
done
if [ $PRECISION = 8 -a "$YUVARG" = "" ]; then
if [ "$ENTROPYARG" = "-optimize" ]; then
IMAGES="vgl_6434_0018a.${EXT}"
elif [ "$ENTROPYARG" = "-progressive" ]; then
IMAGES="vgl_6548_0026a.${EXT}"
elif [ "$ENTROPYARG" = "-arithmetic" -o \
"$ENTROPYARG" = "-progressive-arithmetic" ]; then
IMAGES="shira_bird8.${EXT}"
fi
fi
exec >$EXEDIR/test/tjbenchtest$JAVAARG$YUVARG$ALLOCARG$ENTROPYARG$LOSSLSARG-$PRECISION.log
ALLOCARG=${ALLOCARG//-alloc/-al}
ENTROPYARG=${ENTROPYARG//-optimize/-o}
ENTROPYARG=${ENTROPYARG//-progressive-arithmetic/-pro -a}
ENTROPYARG=${ENTROPYARG//-progressive/-pro}
ENTROPYARG=${ENTROPYARG//-arithmetic/-a}
LOSSLSARG=${LOSSLSARG//-lossless/-l}
XFORMNAME=(hflip vflip transpose transverse rot90 rot180 rot270)
XFORMARGS=("-f h -r 2" "-f v" "-t -r 1b" -transv "-ro 90 -r 1" "-ro 180" "-ro 270")
# Standard tests
for image in $IMAGES; do
cp $IMGDIR/$image $OUTDIR
basename=`basename $image .${EXT}`
r1barg=
if [ "$LOSSLSARG" != "-l" ]; then
r1barg="-r 1b"
fi
runme $EXEDIR/cjpeg -q 95 -pre $PRECISION -dc fa $ENTROPYARG $LOSSLSARG $LOSSLSPSV -gr -outf $OUTDIR/${basename}_GRAY_fast_cjpeg.jpg $IMGDIR/${basename}.${EXT}
runme $EXEDIR/cjpeg -q 95 -pre $PRECISION -dc fa $ENTROPYARG $LOSSLSARG $LOSSLSPSV -sa 1x4 -r 2 -outf $OUTDIR/${basename}_441_fast_cjpeg.jpg $IMGDIR/${basename}.${EXT}
runme $EXEDIR/cjpeg -q 95 -pre $PRECISION -dc fa $ENTROPYARG $LOSSLSARG $LOSSLSPSV -sa 4x1 -r 1 -outf $OUTDIR/${basename}_411_fast_cjpeg.jpg $IMGDIR/${basename}.${EXT}
runme $EXEDIR/cjpeg -q 95 -pre $PRECISION -dc fa $ENTROPYARG $LOSSLSARG $LOSSLSPSV -sa 2x2 -outf $OUTDIR/${basename}_420_fast_cjpeg.jpg $IMGDIR/${basename}.${EXT}
runme $EXEDIR/cjpeg -q 95 -pre $PRECISION -dc fa $ENTROPYARG $LOSSLSARG $LOSSLSPSV -sa 1x2 $r1barg -outf $OUTDIR/${basename}_440_fast_cjpeg.jpg $IMGDIR/${basename}.${EXT}
runme $EXEDIR/cjpeg -q 95 -pre $PRECISION -dc fa $ENTROPYARG $LOSSLSARG $LOSSLSPSV -sa 2x1 -outf $OUTDIR/${basename}_422_fast_cjpeg.jpg $IMGDIR/${basename}.${EXT}
runme $EXEDIR/cjpeg -q 95 -pre $PRECISION -dc fa $ENTROPYARG $LOSSLSARG $LOSSLSPSV -sa 1x1 -outf $OUTDIR/${basename}_444_fast_cjpeg.jpg $IMGDIR/${basename}.${EXT}
runme $EXEDIR/cjpeg -q 95 -pre $PRECISION -dc i $ENTROPYARG $LOSSLSARG $LOSSLSPSV -gr -outf $OUTDIR/${basename}_GRAY_accurate_cjpeg.jpg $IMGDIR/${basename}.${EXT}
runme $EXEDIR/cjpeg -q 95 -pre $PRECISION -dc i $ENTROPYARG $LOSSLSARG $LOSSLSPSV -sa 1x4 -r 2 -outf $OUTDIR/${basename}_441_accurate_cjpeg.jpg $IMGDIR/${basename}.${EXT}
runme $EXEDIR/cjpeg -q 95 -pre $PRECISION -dc i $ENTROPYARG $LOSSLSARG $LOSSLSPSV -sa 4x1 -r 1 -outf $OUTDIR/${basename}_411_accurate_cjpeg.jpg $IMGDIR/${basename}.${EXT}
runme $EXEDIR/cjpeg -q 95 -pre $PRECISION -dc i $ENTROPYARG $LOSSLSARG $LOSSLSPSV -sa 2x2 -outf $OUTDIR/${basename}_420_accurate_cjpeg.jpg $IMGDIR/${basename}.${EXT}
runme $EXEDIR/cjpeg -q 95 -pre $PRECISION -dc i $ENTROPYARG $LOSSLSARG $LOSSLSPSV -sa 1x2 $r1barg -outf $OUTDIR/${basename}_440_accurate_cjpeg.jpg $IMGDIR/${basename}.${EXT}
runme $EXEDIR/cjpeg -q 95 -pre $PRECISION -dc i $ENTROPYARG $LOSSLSARG $LOSSLSPSV -sa 2x1 -outf $OUTDIR/${basename}_422_accurate_cjpeg.jpg $IMGDIR/${basename}.${EXT}
runme $EXEDIR/cjpeg -q 95 -pre $PRECISION -dc i $ENTROPYARG $LOSSLSARG $LOSSLSPSV -sa 1x1 -outf $OUTDIR/${basename}_444_accurate_cjpeg.jpg $IMGDIR/${basename}.${EXT}
for samp in $ALLSUBSAMP; do
runme $EXEDIR/djpeg -dc fa -rg $NSARG $BMPARG -outf $OUTDIR/${basename}_${samp}_fast_djpeg.${EXT} $OUTDIR/${basename}_${samp}_fast_cjpeg.jpg
runme $EXEDIR/djpeg -dc i -rg $NSARG $BMPARG -outf $OUTDIR/${basename}_${samp}_accurate_djpeg.${EXT} $OUTDIR/${basename}_${samp}_accurate_cjpeg.jpg
done
for samp in $x24SUBSAMP; do
runme $EXEDIR/djpeg -dc fa -nos $BMPARG -outf $OUTDIR/${basename}_${samp}_fast_nosmooth_djpeg.${EXT} $OUTDIR/${basename}_${samp}_fast_cjpeg.jpg
runme $EXEDIR/djpeg -dc i -nos $BMPARG -outf $OUTDIR/${basename}_${samp}_accurate_nosmooth_djpeg.${EXT} $OUTDIR/${basename}_${samp}_accurate_cjpeg.jpg
done
# Compression
for dct in accurate fast; do
dctarg=
if [ "${dct}" = "fast" ]; then
dctarg="-d f"
fi
runme $TJBENCH $OUTDIR/$image $TJQUAL -pre $PRECISION -pi rgb -q -be 0.01 -w 0 ${dctarg} $YUVARG $ALLOCARG $ENTROPYARG $LOSSLSARG
if [ "$LOSSLSARG" != "-l" ]; then
runme $TJBENCH $OUTDIR/$image $TJQUAL -pre $PRECISION -su 440 -r 1b -pi rgb -q -be 0.01 -w 0 ${dctarg} $YUVARG $ALLOCARG $ENTROPYARG $LOSSLSARG
runme $TJBENCH $OUTDIR/$image $TJQUAL -pre $PRECISION -su 411 -r 1 -pi rgb -q -be 0.01 -w 0 ${dctarg} $YUVARG $ALLOCARG $ENTROPYARG $LOSSLSARG
runme $TJBENCH $OUTDIR/$image $TJQUAL -pre $PRECISION -su 441 -r 2 -pi rgb -q -be 0.01 -w 0 ${dctarg} $YUVARG $ALLOCARG $ENTROPYARG $LOSSLSARG
fi
for samp in $ALLSUBSAMP; do
if [ "$LOSSLSARG" = "-l" ]; then
runme cmp $OUTDIR/${basename}_LOSSLS_PSV${TJQUAL}.jpg $OUTDIR/${basename}_${samp}_${dct}_cjpeg.jpg
else
runme cmp $OUTDIR/${basename}_${samp}_Q${TJQUAL}.jpg $OUTDIR/${basename}_${samp}_${dct}_cjpeg.jpg
fi
done
done
for dct in fast accurate; do
dctarg=
if [ "${dct}" = "fast" ]; then
dctarg="-d f"
fi
# Tiled compression & decompression
runme $TJBENCH $OUTDIR/$image $TJQUAL -pre $PRECISION -pi rgb -ti -q -be 0.01 -w 0 ${dctarg} $YUVARG $ALLOCARG $ENTROPYARG $LOSSLSARG
for samp in $x1SUBSAMP; do
if [ $ALLOC = 1 ]; then
if [ "$LOSSLSARG" = "-l" ]; then
runme cmp $OUTDIR/${basename}_LOSSLS_PSV${TJQUAL}_full.${EXT} $OUTDIR/${basename}_${samp}_${dct}_djpeg.${EXT}
rm $OUTDIR/${basename}_LOSSLS_PSV${TJQUAL}_full.${EXT}
else
runme cmp $OUTDIR/${basename}_${samp}_Q${TJQUAL}_full.${EXT} $OUTDIR/${basename}_${samp}_${dct}_djpeg.${EXT}
rm $OUTDIR/${basename}_${samp}_Q${TJQUAL}_full.${EXT}
fi
else
if [ "$LOSSLSARG" = "-l" ]; then
for i in $OUTDIR/${basename}_LOSSLS_PSV${TJQUAL}_[0-9]*x[0-9]*.${EXT} \
$OUTDIR/${basename}_LOSSLS_PSV${TJQUAL}_full.${EXT}; do
runme cmp $i $OUTDIR/${basename}_${samp}_${dct}_djpeg.${EXT}
rm $i
done
else
for i in $OUTDIR/${basename}_${samp}_Q${TJQUAL}_[0-9]*x[0-9]*.${EXT} \
$OUTDIR/${basename}_${samp}_Q${TJQUAL}_full.${EXT}; do
runme cmp $i $OUTDIR/${basename}_${samp}_${dct}_djpeg.${EXT}
rm $i
done
fi
fi
done
runme $TJBENCH $OUTDIR/$image $TJQUAL -pre $PRECISION -pi rgb -ti -q -be 0.01 -w 0 -nos ${dctarg} $YUVARG $ALLOCARG $ENTROPYARG $LOSSLSARG
if [ "$LOSSLSARG" != "-l" ]; then
runme $TJBENCH $OUTDIR/$image $TJQUAL -pre $PRECISION -su 440 -pi rgb -ti -q -be 0.01 -w 0 -nos ${dctarg} $YUVARG $ALLOCARG $ENTROPYARG $LOSSLSARG
runme $TJBENCH $OUTDIR/$image $TJQUAL -pre $PRECISION -su 411 -pi rgb -ti -q -be 0.01 -w 0 -nos ${dctarg} $YUVARG $ALLOCARG $ENTROPYARG $LOSSLSARG
runme $TJBENCH $OUTDIR/$image $TJQUAL -pre $PRECISION -su 441 -pi rgb -ti -q -be 0.01 -w 0 -nos ${dctarg} $YUVARG $ALLOCARG $ENTROPYARG $LOSSLSARG
fi
for samp in $x24SUBSAMP; do
if [ $ALLOC = 1 ]; then
if [ "$LOSSLSARG" = "-l" ]; then
runme cmp $OUTDIR/${basename}_LOSSLS_PSV${TJQUAL}_full.${EXT} $OUTDIR/${basename}_${samp}_${dct}_nosmooth_djpeg.${EXT}
rm $OUTDIR/${basename}_LOSSLS_PSV${TJQUAL}_full.${EXT}
else
runme cmp $OUTDIR/${basename}_${samp}_Q${TJQUAL}_full.${EXT} $OUTDIR/${basename}_${samp}_${dct}_nosmooth_djpeg.${EXT}
rm $OUTDIR/${basename}_${samp}_Q${TJQUAL}_full.${EXT}
fi
else
if [ "$LOSSLSARG" = "-l" ]; then
for i in $OUTDIR/${basename}_LOSSLS_PSV${TJQUAL}_[0-9]*x[0-9]*.${EXT} \
$OUTDIR/${basename}_LOSSLS_PSV${TJQUAL}_full.${EXT}; do
runme cmp $i $OUTDIR/${basename}_${samp}_${dct}_nosmooth_djpeg.${EXT}
rm $i
done
else
for i in $OUTDIR/${basename}_${samp}_Q${TJQUAL}_[0-9]*x[0-9]*.${EXT} \
$OUTDIR/${basename}_${samp}_Q${TJQUAL}_full.${EXT}; do
# If the tile size is smaller than the MCU size, then there will be
# edge artifacts at the tile boundaries, so the decompressed image
# will not be identical to the untiled decompressed image.
TILESIZE=$(basename $(echo $i | sed 's/.*_//g') .${EXT})
if [ "$TILESIZE" = "8x8" ]; then
continue
fi
if [ "$TILESIZE" = "16x16" -a \
\( "${samp}" = "411" -o "${samp}" = "441" \) ]; then
continue
fi
runme cmp $i $OUTDIR/${basename}_${samp}_${dct}_nosmooth_djpeg.${EXT}
rm $i
done
fi
fi
done
# Tiled decompression
if [ "$LOSSLSARG" != "-l" ]; then
for samp in $x1SUBSAMP; do
runme $TJBENCH $OUTDIR/${basename}_${samp}_Q95.jpg $BMPARG -ti -q -be 0.01 -w 0 ${dctarg} $YUVARG $ALLOCARG $ENTROPYARG
if [ $ALLOC = 1 ]; then
runme cmp $OUTDIR/${basename}_${samp}_Q95_full.${EXT} $OUTDIR/${basename}_${samp}_${dct}_djpeg.${EXT}
rm $OUTDIR/${basename}_${samp}_Q95_full.${EXT}
else
for i in $OUTDIR/${basename}_${samp}_Q95_[0-9]*x[0-9]*.${EXT} \
$OUTDIR/${basename}_${samp}_Q95_full.${EXT}; do
runme cmp $i $OUTDIR/${basename}_${samp}_${dct}_djpeg.${EXT}
rm $i
done
fi
done
for samp in $x24SUBSAMP; do
runme $TJBENCH $OUTDIR/${basename}_${samp}_Q95.jpg $BMPARG -ti -q -be 0.01 -w 0 -nos ${dctarg} $YUVARG $ALLOCARG $ENTROPYARG
if [ $ALLOC = 1 ]; then
runme cmp $OUTDIR/${basename}_${samp}_Q95_full.${EXT} $OUTDIR/${basename}_${samp}_${dct}_nosmooth_djpeg.${EXT}
rm $OUTDIR/${basename}_${samp}_Q95_full.${EXT}
else
for i in $OUTDIR/${basename}_${samp}_Q95_[0-9]*x[0-9]*.${EXT} \
$OUTDIR/${basename}_${samp}_Q95_full.${EXT}; do
TILESIZE=$(basename $(echo $i | sed 's/.*_//g') .${EXT})
if [ "$TILESIZE" = "8x8" ]; then
continue
fi
if [ "$TILESIZE" = "16x16" -a \
\( "${samp}" = "411" -o "${samp}" = "441" \) ]; then
continue
fi
runme cmp $i $OUTDIR/${basename}_${samp}_${dct}_nosmooth_djpeg.${EXT}
rm $i
done
fi
done
fi
done
# Partial decompression
if [ "$LOSSLSARG" != "-l" -a "$YUVARG" != "-yuv" ]; then
for samp in $ALLSUBSAMP; do
CROPW8_8=103
CROPL8_8=16
CROPW7_8=91
CROPL7_8=14
if [ "${samp}" = "411" ]; then
CROPW8_8=87
CROPL8_8=32
CROPW7_8=77
CROPL7_8=28
fi
runme $EXEDIR/djpeg -rg -cr ${CROPW8_8}x90+${CROPL8_8}+5 $NSARG -outf $OUTDIR/${basename}_${samp}_scale_crop_djpeg.ppm $OUTDIR/${basename}_${samp}_accurate_cjpeg.jpg
runme $TJBENCH $OUTDIR/${basename}_${samp}_Q${TJQUAL}.jpg -cr ${CROPW8_8}X90+${CROPL8_8}+5 -q -be 0.01 -w 0 ${dctarg} $YUVARG $ALLOCARG $ENTROPYARG $LOSSLSARG
runme cmp $OUTDIR/${basename}_${samp}_Q${TJQUAL}_full.ppm $OUTDIR/${basename}_${samp}_scale_crop_djpeg.ppm
rm $OUTDIR/${basename}_${samp}_Q${TJQUAL}_full.ppm $OUTDIR/${basename}_${samp}_scale_crop_djpeg.ppm
runme $EXEDIR/djpeg -rg -s 14/16 -cr ${CROPW7_8}X81+${CROPL7_8}+3 $NSARG -outf $OUTDIR/${basename}_${samp}_scale_crop_djpeg.ppm $OUTDIR/${basename}_${samp}_accurate_cjpeg.jpg
runme $TJBENCH $OUTDIR/${basename}_${samp}_Q${TJQUAL}.jpg -s 7/8 -cr ${CROPW7_8}x81+${CROPL7_8}+3 -q -be 0.01 -w 0 ${dctarg} $YUVARG $ALLOCARG $ENTROPYARG $LOSSLSARG
runme cmp $OUTDIR/${basename}_${samp}_Q${TJQUAL}_7_8.ppm $OUTDIR/${basename}_${samp}_scale_crop_djpeg.ppm
rm $OUTDIR/${basename}_${samp}_Q${TJQUAL}_7_8.ppm $OUTDIR/${basename}_${samp}_scale_crop_djpeg.ppm
runme $EXEDIR/djpeg -rg -s 1/2 -cr 40x40+0+0 $NSARG -outf $OUTDIR/${basename}_${samp}_scale_crop_djpeg.ppm $OUTDIR/${basename}_${samp}_accurate_cjpeg.jpg
runme $TJBENCH $OUTDIR/${basename}_${samp}_Q${TJQUAL}.jpg -s 4/8 -cr 40x40+0+0 -q -be 0.01 -w 0 ${dctarg} $YUVARG $ALLOCARG $ENTROPYARG $LOSSLSARG
runme cmp $OUTDIR/${basename}_${samp}_Q${TJQUAL}_1_2.ppm $OUTDIR/${basename}_${samp}_scale_crop_djpeg.ppm
rm $OUTDIR/${basename}_${samp}_Q${TJQUAL}_1_2.ppm $OUTDIR/${basename}_${samp}_scale_crop_djpeg.ppm
done
fi
# Scaled decompression
for scale in 2_1 15_8 7_4 13_8 3_2 11_8 5_4 9_8 7_8 3_4 5_8 1_2 3_8 1_4 1_8; do
scalearg=`echo $scale | sed 's/\_/\//g'`
SCALE=$scale
if [ "$LOSSLSARG" = "-l" ]; then
SCALE=full
fi
for samp in $ALLSUBSAMP; do
runme $EXEDIR/djpeg -rg -s ${scalearg} $NSARG $BMPARG -outf $OUTDIR/${basename}_${samp}_${scale}_djpeg.${EXT} $OUTDIR/${basename}_${samp}_accurate_cjpeg.jpg
if [ "$LOSSLSARG" = "-l" ]; then
runme $TJBENCH $OUTDIR/${basename}_LOSSLS_PSV${TJQUAL}.jpg $BMPARG -s ${scalearg} -q -be 0.01 -w 0 $YUVARG $ALLOCARG $ENTROPYARG $LOSSLSARG
runme cmp $OUTDIR/${basename}_LOSSLS_PSV${TJQUAL}_${SCALE}.${EXT} $OUTDIR/${basename}_${samp}_${scale}_djpeg.${EXT}
rm $OUTDIR/${basename}_LOSSLS_PSV${TJQUAL}_${SCALE}.${EXT}
else
runme $TJBENCH $OUTDIR/${basename}_${samp}_Q${TJQUAL}.jpg $BMPARG -s ${scalearg} -q -be 0.01 -w 0 $YUVARG $ALLOCARG $ENTROPYARG $LOSSLSARG
runme cmp $OUTDIR/${basename}_${samp}_Q${TJQUAL}_${SCALE}.${EXT} $OUTDIR/${basename}_${samp}_${scale}_djpeg.${EXT}
rm $OUTDIR/${basename}_${samp}_Q${TJQUAL}_${SCALE}.${EXT}
fi
done
done
# Transforms
if [ "$LOSSLSARG" != "-l" ]; then
runme $EXEDIR/jpegtran -icc $IMGDIR/test1.icc -outf $OUTDIR/temp.jpg $OUTDIR/${basename}_444_Q95.jpg
runme mv $OUTDIR/temp.jpg $OUTDIR/${basename}_444_Q95.jpg
for samp in $ALLSUBSAMP; do
for xform in {0..6}; do
runme $EXEDIR/jpegtran -c a ${XFORMARGS[$xform]} -tri -outf $OUTDIR/${basename}_${samp}_${XFORMNAME[$xform]}_jpegtran.jpg $OUTDIR/${basename}_${samp}_Q95.jpg
done
done
for xform in {0..6}; do
for samp in $x1SUBSAMP; do
runme $EXEDIR/djpeg -rg $BMPARG -outf $OUTDIR/${basename}_${samp}_${XFORMNAME[$xform]}_jpegtran.${EXT} $OUTDIR/${basename}_${samp}_${XFORMNAME[$xform]}_jpegtran.jpg
runme $TJBENCH $OUTDIR/${basename}_${samp}_Q95.jpg $BMPARG ${XFORMARGS[$xform]} -ti -q -be 0.01 -w 0 $YUVARG $ALLOCARG $ENTROPYARG
if [ $ALLOC = 1 ]; then
runme cmp $OUTDIR/${basename}_${samp}_Q95_full.${EXT} $OUTDIR/${basename}_${samp}_${XFORMNAME[$xform]}_jpegtran.${EXT}
rm $OUTDIR/${basename}_${samp}_Q95_full.${EXT}
else
for i in $OUTDIR/${basename}_${samp}_Q95_[0-9]*x[0-9]*.${EXT} \
$OUTDIR/${basename}_${samp}_Q95_full.${EXT}; do
runme cmp $i $OUTDIR/${basename}_${samp}_${XFORMNAME[$xform]}_jpegtran.${EXT}
rm $i
done
fi
done
for samp in $x24SUBSAMP; do
runme $EXEDIR/djpeg -nos -rg $BMPARG -outf $OUTDIR/${basename}_${samp}_${XFORMNAME[$xform]}_jpegtran.${EXT} $OUTDIR/${basename}_${samp}_${XFORMNAME[$xform]}_jpegtran.jpg
runme $TJBENCH $OUTDIR/${basename}_${samp}_Q95.jpg $BMPARG ${XFORMARGS[$xform]} -ti -q -be 0.01 -w 0 -nos $YUVARG $ALLOCARG $ENTROPYARG
if [ $ALLOC = 1 ]; then
runme cmp $OUTDIR/${basename}_${samp}_Q95_full.${EXT} $OUTDIR/${basename}_${samp}_${XFORMNAME[$xform]}_jpegtran.${EXT}
rm $OUTDIR/${basename}_${samp}_Q95_full.${EXT}
else
for i in $OUTDIR/${basename}_${samp}_Q95_[0-9]*x[0-9]*.${EXT} \
$OUTDIR/${basename}_${samp}_Q95_full.${EXT}; do
TILESIZE=$(basename $(echo $i | sed 's/.*_//g') .${EXT})
if [ "$TILESIZE" = "8x8" ]; then
continue
fi
if [ "$TILESIZE" = "16x16" -a \
\( "${samp}" = "411" -o "${samp}" = "441" \) ]; then
continue
fi
runme cmp $i $OUTDIR/${basename}_${samp}_${XFORMNAME[$xform]}_jpegtran.${EXT}
rm $i
done
fi
done
done
# Grayscale transform
for xform in {0..6}; do
for samp in $ALLSUBSAMP; do
runme $TJBENCH $OUTDIR/${basename}_${samp}_Q95.jpg $BMPARG ${XFORMARGS[$xform]} -ti -q -be 0.01 -w 0 -g $YUVARG $ALLOCARG $ENTROPYARG
if [ $ALLOC = 1 ]; then
runme cmp $OUTDIR/${basename}_${samp}_Q95_full.${EXT} $OUTDIR/${basename}_GRAY_${XFORMNAME[$xform]}_jpegtran.${EXT}
rm $OUTDIR/${basename}_${samp}_Q95_full.${EXT}
else
for i in $OUTDIR/${basename}_${samp}_Q95_[0-9]*x[0-9]*.${EXT} \
$OUTDIR/${basename}_${samp}_Q95_full.${EXT}; do
TILESIZE=$(basename $(echo $i | sed 's/.*_//g') .${EXT})
if [ "$TILESIZE" = "8x8" -a \
"${samp}" != "444" -a "${samp}" != "GRAY" ]; then
continue
fi
if [ "$TILESIZE" = "16x16" -a \
\( "${samp}" = "411" -o "${samp}" = "441" \) ]; then
continue
fi
runme cmp $i $OUTDIR/${basename}_GRAY_${XFORMNAME[$xform]}_jpegtran.${EXT}
rm $i
done
fi
done
done
# Transforms with scaling
for xform in {0..6}; do
for samp in $ALLSUBSAMP; do
for scale in 2_1 15_8 7_4 13_8 3_2 11_8 5_4 9_8 7_8 3_4 5_8 1_2 3_8 1_4 1_8; do
scalearg=`echo $scale | sed 's/\_/\//g'`
runme $EXEDIR/djpeg -rg -s ${scalearg} $NSARG $BMPARG -outf $OUTDIR/${basename}_${samp}_${XFORMNAME[$xform]}_${scale}_jpegtran.${EXT} $OUTDIR/${basename}_${samp}_${XFORMNAME[$xform]}_jpegtran.jpg
runme $TJBENCH $OUTDIR/${basename}_${samp}_Q95.jpg $BMPARG ${XFORMARGS[$xform]} -s ${scalearg} -q -be 0.01 -w 0 $YUVARG $ALLOCARG $ENTROPYARG
runme cmp $OUTDIR/${basename}_${samp}_Q95_${scale}.${EXT} $OUTDIR/${basename}_${samp}_${XFORMNAME[$xform]}_${scale}_jpegtran.${EXT}
rm $OUTDIR/${basename}_${samp}_Q95_${scale}.${EXT}
done
done
done
fi
done
echo SUCCESS!
lib/libjpeg-turbo/test/tjcomptest.in
Executable
+195
View File
@@ -0,0 +1,195 @@
#/bin/bash
set -u
set -e
trap onexit INT
trap onexit TERM
trap onexit EXIT
onexit()
{
if [ -d $OUTDIR ]; then
rm -rf $OUTDIR
fi
}
runme()
{
echo \*\*\* $* | sed "s@$EXEDIR@\$EXEDIR@g" | sed "s@$IMGDIR@\$IMGDIR@g" |
sed "s@$JAVA@\$JAVA@g" | sed "s@$OUTDIR@\$OUTDIR@g"
"$@"
}
IMGDIR=@CMAKE_SOURCE_DIR@/testimages
OUTDIR=`mktemp -d /tmp/__tjcomptest_output.XXXXXX`
EXEDIR=@CMAKE_BINARY_DIR@
JAVA="@Java_JAVA_EXECUTABLE@"
JAVAARGS="-cp $EXEDIR/java/turbojpeg.jar -Djava.library.path=$EXEDIR"
TJCOMP=$EXEDIR/tjcomp
JAVAARG=
if [ -d $OUTDIR ]; then
rm -rf $OUTDIR
fi
mkdir -p $OUTDIR
while [ $# -gt 0 ]; do
case "$1" in
-java)
JAVAARG=-java
TJCOMP="$JAVA $JAVAARGS TJComp"
;;
esac
shift
done
exec >$EXEDIR/test/tjcomptest$JAVAARG.log
echo "EXEDIR: $EXEDIR"
echo "IMGDIR: $IMGDIR"
echo "OUTDIR: $OUTDIR"
echo
SUBSAMPOPT=(444 422 440 420 411 441)
SAMPOPT=(1x1 2x1 1x2 2x2 4x1 1x4)
for precision in 8 12; do
if [ $precision -le 8 ]; then
RGBIMG=$IMGDIR/testorig.ppm
GRAYIMG=$IMGDIR/testorig.pgm
else
RGBIMG=$IMGDIR/monkey16.ppm
GRAYIMG=$IMGDIR/monkey16.pgm
fi
if [ $precision -eq 8 ]; then
prearg=
else
prearg="-pre $precision"
fi
for restartarg in "" "-r 1 -icc $IMGDIR/test3.icc" "-r 1b"; do
for ariarg in "" "-a"; do
for dctarg in "" "-dc fa"; do
for optarg in "" "-o"; do
if [ "$optarg" = "-o" ]; then
if [[ "$ariarg" = "-a" || $precision -eq 12 ]]; then
continue
fi
fi
for progarg in "" "-p"; do
if [[ "$progarg" = "-p" && "$optarg" = "-o" ]]; then
continue
fi
for qualarg in "" "-q 1" "-q 100"; do
blarg=
if [ "$qualarg" = "-q 1" ]; then
blarg=-baseline
fi
for sampi in {0..5}; do
basename=`basename $RGBIMG .ppm`
runme $TJCOMP $prearg $restartarg $ariarg $dctarg \
$optarg $progarg $qualarg -s ${SUBSAMPOPT[$sampi]} \
$RGBIMG $OUTDIR/${basename}-tjcomp.jpg
runme $EXEDIR/cjpeg $prearg $restartarg $ariarg $dctarg \
$optarg $progarg $qualarg $blarg -sa ${SAMPOPT[$sampi]} \
-outf $OUTDIR/${basename}-cjpeg.jpg $RGBIMG
$EXEDIR/test/md5sum $OUTDIR/${basename}-tjcomp.jpg \
$OUTDIR/${basename}-cjpeg.jpg | sed "s@$OUTDIR@\$OUTDIR@g"
cmp $OUTDIR/${basename}-tjcomp.jpg \
$OUTDIR/${basename}-cjpeg.jpg
rm $OUTDIR/${basename}-tjcomp.jpg $OUTDIR/${basename}-cjpeg.jpg
echo
runme $TJCOMP $prearg $restartarg $ariarg $dctarg \
$optarg $progarg $qualarg -s ${SUBSAMPOPT[$sampi]} \
-g $RGBIMG $OUTDIR/${basename}-tjcomp.jpg
runme $EXEDIR/cjpeg $prearg $restartarg $ariarg $dctarg \
$optarg $progarg $qualarg $blarg -sa ${SAMPOPT[$sampi]} \
-gr -outf $OUTDIR/${basename}-cjpeg.jpg $RGBIMG
$EXEDIR/test/md5sum $OUTDIR/${basename}-tjcomp.jpg \
$OUTDIR/${basename}-cjpeg.jpg | sed "s@$OUTDIR@\$OUTDIR@g"
cmp $OUTDIR/${basename}-tjcomp.jpg \
$OUTDIR/${basename}-cjpeg.jpg
rm $OUTDIR/${basename}-tjcomp.jpg $OUTDIR/${basename}-cjpeg.jpg
echo
runme $TJCOMP $prearg $restartarg $ariarg $dctarg \
$optarg $progarg $qualarg -s ${SUBSAMPOPT[$sampi]} \
-rg $RGBIMG $OUTDIR/${basename}-tjcomp.jpg
runme $EXEDIR/cjpeg $prearg $restartarg $ariarg $dctarg \
$optarg $progarg $qualarg $blarg -sa ${SAMPOPT[$sampi]} \
-rgb -outf $OUTDIR/${basename}-cjpeg.jpg $RGBIMG
$EXEDIR/test/md5sum $OUTDIR/${basename}-tjcomp.jpg \
$OUTDIR/${basename}-cjpeg.jpg | sed "s@$OUTDIR@\$OUTDIR@g"
cmp $OUTDIR/${basename}-tjcomp.jpg \
$OUTDIR/${basename}-cjpeg.jpg
rm $OUTDIR/${basename}-tjcomp.jpg $OUTDIR/${basename}-cjpeg.jpg
echo
basename=`basename $GRAYIMG .pgm`
runme $TJCOMP $prearg $restartarg $ariarg $dctarg \
$optarg $progarg $qualarg -s ${SUBSAMPOPT[$sampi]} \
$GRAYIMG $OUTDIR/${basename}-tjcomp.jpg
runme $EXEDIR/cjpeg $prearg $restartarg $ariarg $dctarg \
$optarg $progarg $qualarg $blarg -sa ${SAMPOPT[$sampi]} \
-outf $OUTDIR/${basename}-cjpeg.jpg $GRAYIMG
$EXEDIR/test/md5sum $OUTDIR/${basename}-tjcomp.jpg \
$OUTDIR/${basename}-cjpeg.jpg | sed "s@$OUTDIR@\$OUTDIR@g"
cmp $OUTDIR/${basename}-tjcomp.jpg \
$OUTDIR/${basename}-cjpeg.jpg
rm $OUTDIR/${basename}-tjcomp.jpg $OUTDIR/${basename}-cjpeg.jpg
echo
done
done
done
done
done
done
done
done
for precision in {2..16}; do
if [ $precision -le 8 ]; then
RGBIMG=$IMGDIR/testorig.ppm
GRAYIMG=$IMGDIR/testorig.pgm
else
RGBIMG=$IMGDIR/monkey16.ppm
GRAYIMG=$IMGDIR/monkey16.pgm
fi
if [ $precision -eq 8 ]; then
prearg=
else
prearg="-pre $precision"
fi
for psv in {1..7}; do
for pt in {0..15}; do
if [ $pt -ge $precision ]; then
continue
fi
for restartarg in "" "-r 1 -icc $IMGDIR/test3.icc"; do
basename=`basename $RGBIMG .ppm`
runme $TJCOMP $prearg -l $psv,$pt $restartarg \
$RGBIMG $OUTDIR/${basename}-tjcomp.jpg
runme $EXEDIR/cjpeg $prearg -l $psv,$pt $restartarg \
-outf $OUTDIR/${basename}-cjpeg.jpg $RGBIMG
$EXEDIR/test/md5sum $OUTDIR/${basename}-tjcomp.jpg \
$OUTDIR/${basename}-cjpeg.jpg | sed "s@$OUTDIR@\$OUTDIR@g"
cmp $OUTDIR/${basename}-tjcomp.jpg $OUTDIR/${basename}-cjpeg.jpg
rm $OUTDIR/${basename}-tjcomp.jpg $OUTDIR/${basename}-cjpeg.jpg
echo
basename=`basename $GRAYIMG .pgm`
runme $TJCOMP $prearg -l $psv,$pt $restartarg \
$GRAYIMG $OUTDIR/${basename}-tjcomp.jpg
runme $EXEDIR/cjpeg $prearg -l $psv,$pt $restartarg \
-outf $OUTDIR/${basename}-cjpeg.jpg $GRAYIMG
$EXEDIR/test/md5sum $OUTDIR/${basename}-tjcomp.jpg \
$OUTDIR/${basename}-cjpeg.jpg | sed "s@$OUTDIR@\$OUTDIR@g"
cmp $OUTDIR/${basename}-tjcomp.jpg $OUTDIR/${basename}-cjpeg.jpg
rm $OUTDIR/${basename}-tjcomp.jpg $OUTDIR/${basename}-cjpeg.jpg
echo
done
done
done
done
echo "GREAT SUCCESS!"
lib/libjpeg-turbo/test/tjdecomptest.in
Executable
+224
View File
@@ -0,0 +1,224 @@
#/bin/bash
set -u
set -e
trap onexit INT
trap onexit TERM
trap onexit EXIT
onexit()
{
if [ -d $OUTDIR ]; then
rm -rf $OUTDIR
fi
}
runme()
{
echo \*\*\* $* | sed "s@$EXEDIR@\$EXEDIR@g" | sed "s@$IMGDIR@\$IMGDIR@g" |
sed "s@$JAVA@\$JAVA@g" | sed "s@$OUTDIR@\$OUTDIR@g"
"$@"
}
IMGDIR=@CMAKE_SOURCE_DIR@/testimages
OUTDIR=`mktemp -d /tmp/__tjdecomptest_output.XXXXXX`
EXEDIR=@CMAKE_BINARY_DIR@
JAVA="@Java_JAVA_EXECUTABLE@"
JAVAARGS="-cp $EXEDIR/java/turbojpeg.jar -Djava.library.path=$EXEDIR"
TJDECOMP=$EXEDIR/tjdecomp
JAVAARG=
if [ -d $OUTDIR ]; then
rm -rf $OUTDIR
fi
mkdir -p $OUTDIR
while [ $# -gt 0 ]; do
case "$1" in
-java)
JAVAARG=-java
TJDECOMP="$JAVA $JAVAARGS TJDecomp"
;;
esac
shift
done
exec >$EXEDIR/test/tjdecomptest$JAVAARG.log
echo "EXEDIR: $EXEDIR"
echo "IMGDIR: $IMGDIR"
echo "OUTDIR: $OUTDIR"
echo
SUBSAMPOPT=(444 422 440 420 411 441 410)
SAMPOPT=(1x1 2x1 1x2 2x2 4x1 1x4 4x2)
for precision in 8 12; do
if [ $precision -le 8 ]; then
RGBIMG=$IMGDIR/testorig.ppm
GRAYIMG=$IMGDIR/testorig.pgm
else
RGBIMG=$IMGDIR/monkey16.ppm
GRAYIMG=$IMGDIR/monkey16.pgm
fi
for sampi in {0..6}; do
iccarg=
if [ $sampi = 3 ]; then
iccarg="-icc $IMGDIR/test1.icc"
fi
runme $EXEDIR/cjpeg -pre $precision -sa ${SAMPOPT[$sampi]} $iccarg \
-outf $OUTDIR/`basename $RGBIMG .ppm`-${SUBSAMPOPT[$sampi]}.jpg $RGBIMG
done
runme $EXEDIR/cjpeg -pre $precision \
-outf $OUTDIR/`basename $GRAYIMG .pgm`-gray.jpg $GRAYIMG
echo
for subsamp in ${SUBSAMPOPT[*]} gray; do
for croparg in "" "-cr 14x14+23+23" "-cr 21x21+4+4" "-cr 18x18+13+13" \
"-cr 21x21+0+0" "-cr 24x26+20+18"; do
if [[ "$croparg" != "" && "$subsamp" = "410" ]]; then
continue
fi
for scalearg in "" "-s 16/8" "-s 15/8" "-s 14/8" "-s 13/8" "-s 12/8" \
"-s 11/8" "-s 10/8" "-s 9/8" "-s 7/8" "-s 6/8" "-s 5/8" "-s 4/8" \
"-s 3/8" "-s 2/8" "-s 1/8"; do
if [[ ("$scalearg" = "-s 1/8" || "$scalearg" = "-s 2/8" || \
"$scalearg" = "-s 3/8") && "$croparg" != "" ]]; then
continue
fi
for nsarg in "" "-nos"; do
if [[ "$nsarg" = "-nos" && "$subsamp" != "422" && \
"$subsamp" != "420" && "$subsamp" != "440" ]]; then
continue
fi
for dctarg in "" "-dc fa"; do
if [[ "$dctarg" = "-dc fa" && \
("$scalearg" != "-s 4/8" || \
("$subsamp" != "420" && "$subsamp" != "410")) && \
"$scalearg" != "" ]]; then
continue
fi
if [ "$subsamp" = "gray" ]; then
basename=`basename $GRAYIMG .pgm`
runme $TJDECOMP $croparg $dctarg $nsarg $scalearg \
$OUTDIR/${basename}-$subsamp.jpg \
$OUTDIR/${basename}-tjdecomp.pgm
runme $EXEDIR/djpeg $croparg $dctarg $nsarg $scalearg \
-outf $OUTDIR/${basename}-djpeg.pgm \
$OUTDIR/${basename}-$subsamp.jpg
$EXEDIR/test/md5sum $OUTDIR/${basename}-tjdecomp.pgm \
$OUTDIR/${basename}-djpeg.pgm | sed "s@$OUTDIR@\$OUTDIR@g"
cmp $OUTDIR/${basename}-tjdecomp.pgm \
$OUTDIR/${basename}-djpeg.pgm
rm $OUTDIR/${basename}-tjdecomp.pgm $OUTDIR/${basename}-djpeg.pgm
echo
runme $TJDECOMP $croparg $dctarg $nsarg $scalearg \
-r $OUTDIR/${basename}-$subsamp.jpg \
$OUTDIR/${basename}-tjdecomp.ppm
runme $EXEDIR/djpeg $croparg $dctarg $nsarg $scalearg \
-rg -outf $OUTDIR/${basename}-djpeg.ppm \
$OUTDIR/${basename}-$subsamp.jpg
$EXEDIR/test/md5sum $OUTDIR/${basename}-tjdecomp.ppm \
$OUTDIR/${basename}-djpeg.ppm | sed "s@$OUTDIR@\$OUTDIR@g"
cmp $OUTDIR/${basename}-tjdecomp.ppm \
$OUTDIR/${basename}-djpeg.ppm
rm $OUTDIR/${basename}-tjdecomp.ppm $OUTDIR/${basename}-djpeg.ppm
echo
else
basename=`basename $RGBIMG .ppm`
iccarg=
if [ "$subsamp" = "420" ]; then
iccarg="-icc $OUTDIR/${basename}-tjdecomp.icc"
fi
runme $TJDECOMP $croparg $dctarg $nsarg $scalearg $iccarg \
$OUTDIR/${basename}-$subsamp.jpg \
$OUTDIR/${basename}-tjdecomp.ppm
if [ "$subsamp" = "420" ]; then
iccarg="-icc $OUTDIR/${basename}-djpeg.icc"
fi
runme $EXEDIR/djpeg $croparg $dctarg $nsarg $scalearg $iccarg \
-outf $OUTDIR/${basename}-djpeg.ppm \
$OUTDIR/${basename}-$subsamp.jpg
if [ "$subsamp" = "420" ]; then
$EXEDIR/test/md5sum $OUTDIR/${basename}-tjdecomp.icc \
$OUTDIR/${basename}-djpeg.icc | sed "s@$OUTDIR@\$OUTDIR@g"
cmp $OUTDIR/${basename}-tjdecomp.icc \
$OUTDIR/${basename}-djpeg.icc
rm $OUTDIR/${basename}-tjdecomp.icc $OUTDIR/${basename}-djpeg.icc
fi
$EXEDIR/test/md5sum $OUTDIR/${basename}-tjdecomp.ppm \
$OUTDIR/${basename}-djpeg.ppm | sed "s@$OUTDIR@\$OUTDIR@g"
cmp $OUTDIR/${basename}-tjdecomp.ppm \
$OUTDIR/${basename}-djpeg.ppm
rm $OUTDIR/${basename}-tjdecomp.ppm $OUTDIR/${basename}-djpeg.ppm
echo
if [[ "$nsarg" = "" ]]; then
runme $TJDECOMP $croparg $dctarg $nsarg $scalearg \
-g $OUTDIR/${basename}-$subsamp.jpg \
$OUTDIR/${basename}-tjdecomp.pgm
runme $EXEDIR/djpeg $croparg $dctarg $nsarg $scalearg \
-gr -outf $OUTDIR/${basename}-djpeg.pgm \
$OUTDIR/${basename}-$subsamp.jpg
$EXEDIR/test/md5sum $OUTDIR/${basename}-tjdecomp.pgm \
$OUTDIR/${basename}-djpeg.pgm | sed "s@$OUTDIR@\$OUTDIR@g"
cmp $OUTDIR/${basename}-tjdecomp.pgm \
$OUTDIR/${basename}-djpeg.pgm
rm $OUTDIR/${basename}-tjdecomp.pgm \
$OUTDIR/${basename}-djpeg.pgm
echo
fi
fi
done
done
done
done
rm $OUTDIR/${basename}-$subsamp.jpg
done
done
for precision in {2..16}; do
if [ $precision -le 8 ]; then
RGBIMG=$IMGDIR/testorig.ppm
GRAYIMG=$IMGDIR/testorig.pgm
else
RGBIMG=$IMGDIR/monkey16.ppm
GRAYIMG=$IMGDIR/monkey16.pgm
fi
basename=`basename $RGBIMG .ppm`
runme $EXEDIR/cjpeg -pre $precision -l 1 \
-outf $OUTDIR/${basename}-rgb.jpg $RGBIMG
echo
runme $TJDECOMP $OUTDIR/${basename}-rgb.jpg \
$OUTDIR/${basename}-tjdecomp.ppm
runme $EXEDIR/djpeg -outf $OUTDIR/${basename}-djpeg.ppm \
$OUTDIR/${basename}-rgb.jpg
$EXEDIR/test/md5sum $OUTDIR/${basename}-tjdecomp.ppm \
$OUTDIR/${basename}-djpeg.ppm | sed "s@$OUTDIR@\$OUTDIR@g"
cmp $OUTDIR/${basename}-tjdecomp.ppm $OUTDIR/${basename}-djpeg.ppm
rm $OUTDIR/${basename}-tjdecomp.ppm $OUTDIR/${basename}-djpeg.ppm
rm $OUTDIR/${basename}-rgb.jpg
echo
basename=`basename $GRAYIMG .pgm`
runme $EXEDIR/cjpeg -pre $precision -l 1 \
-outf $OUTDIR/${basename}-gray.jpg $GRAYIMG
echo
runme $TJDECOMP $OUTDIR/${basename}-gray.jpg \
$OUTDIR/${basename}-tjdecomp.pgm
runme $EXEDIR/djpeg -outf $OUTDIR/${basename}-djpeg.pgm \
$OUTDIR/${basename}-gray.jpg
$EXEDIR/test/md5sum $OUTDIR/${basename}-tjdecomp.pgm \
$OUTDIR/${basename}-djpeg.pgm | sed "s@$OUTDIR@\$OUTDIR@g"
cmp $OUTDIR/${basename}-tjdecomp.pgm $OUTDIR/${basename}-djpeg.pgm
rm $OUTDIR/${basename}-tjdecomp.pgm $OUTDIR/${basename}-djpeg.pgm
rm $OUTDIR/${basename}-gray.jpg
echo
done
echo "GREAT SUCCESS!"
lib/libjpeg-turbo/test/tjtrantest.in
Executable
+172
View File
@@ -0,0 +1,172 @@
#/bin/bash
set -u
set -e
trap onexit INT
trap onexit TERM
trap onexit EXIT
onexit()
{
if [ -d $OUTDIR ]; then
rm -rf $OUTDIR
fi
}
runme()
{
echo \*\*\* $* | sed "s@$EXEDIR@\$EXEDIR@g" | sed "s@$IMGDIR@\$IMGDIR@g" |
sed "s@$JAVA@\$JAVA@g" | sed "s@$OUTDIR@\$OUTDIR@g"
"$@"
}
IMGDIR=@CMAKE_SOURCE_DIR@/testimages
OUTDIR=`mktemp -d /tmp/__tjtrantest_output.XXXXXX`
EXEDIR=@CMAKE_BINARY_DIR@
JAVA="@Java_JAVA_EXECUTABLE@"
JAVAARGS="-cp $EXEDIR/java/turbojpeg.jar -Djava.library.path=$EXEDIR"
TJTRAN=$EXEDIR/tjtran
JAVAARG=
if [ -d $OUTDIR ]; then
rm -rf $OUTDIR
fi
mkdir -p $OUTDIR
while [ $# -gt 0 ]; do
case "$1" in
-java)
JAVAARG=-java
TJTRAN="$JAVA $JAVAARGS TJTran"
;;
esac
shift
done
exec >$EXEDIR/test/tjtrantest$JAVAARG.log
echo "EXEDIR: $EXEDIR"
echo "IMGDIR: $IMGDIR"
echo "OUTDIR: $OUTDIR"
echo
SUBSAMPOPT=(444 422 440 420 411 441 410)
SAMPOPT=(1x1 2x1 1x2 2x2 4x1 1x4 4x2)
for precision in 8 12; do
if [ $precision -le 8 ]; then
RGBIMG=$IMGDIR/testorig.ppm
GRAYIMG=$IMGDIR/testorig.pgm
else
RGBIMG=$IMGDIR/monkey16.ppm
GRAYIMG=$IMGDIR/monkey16.pgm
fi
for sampi in {0..6}; do
EXTRA_ARGS=
if [ $sampi = 1 ]; then
EXTRA_ARGS=-p
elif [ $sampi = 2 ]; then
EXTRA_ARGS=-a
elif [ $sampi = 3 ]; then
EXTRA_ARGS="-o -icc $IMGDIR/test1.icc"
elif [ $sampi = 5 ]; then
EXTRA_ARGS="-r 1"
fi
runme $EXEDIR/cjpeg -pre $precision -sa ${SAMPOPT[$sampi]} $EXTRA_ARGS \
-outf $OUTDIR/`basename $RGBIMG .ppm`-${SUBSAMPOPT[$sampi]}.jpg $RGBIMG
if [[ $sampi = 3 || $sampi = 4 ]]; then
$EXEDIR/wrjpgcom -comment "This is a test" \
$OUTDIR/`basename $RGBIMG .ppm`-${SUBSAMPOPT[$sampi]}.jpg \
>$OUTDIR/temp.jpg
mv $OUTDIR/temp.jpg \
$OUTDIR/`basename $RGBIMG .ppm`-${SUBSAMPOPT[$sampi]}.jpg
fi
done
runme $EXEDIR/cjpeg -pre $precision \
-outf $OUTDIR/`basename $GRAYIMG .pgm`-gray.jpg $GRAYIMG
echo
for subsamp in ${SUBSAMPOPT[*]} gray; do
if [ "$subsamp" = "gray" ]; then
basename=`basename $GRAYIMG .pgm`
else
basename=`basename $RGBIMG .ppm`
fi
for ariarg in "" "-a"; do
for copyarg in "" "-c i" "-c n"; do
if [[ "$copyarg" = "-c n" && "$subsamp" != "411" && \
"$subsamp" != "420" ]]; then
continue
fi
if [[ "$copyarg" = "-c i" && "$subsamp" != "420" ]]; then
continue
fi
for croparg in "" "-cr 14x14+23+23" "-cr 21x21+4+4" "-cr 18x18+13+13" \
"-cr 21x21+0+0" "-cr 24x26+20+18"; do
for xformarg in "" "-f h" "-f v" "-ro 90" "-ro 180" "-ro 270" "-t" \
"-transv"; do
for grayarg in "" "-g"; do
if [ "$grayarg" = "" ]; then
if [[ "$subsamp" = "410" && "$croparg" != "" ]]; then
continue
fi
else
if [ "$subsamp" = "gray" ]; then
continue
fi
fi
for optarg in "" "-o"; do
if [ "$optarg" = "-o" ]; then
if [[ "$ariarg" = "-a" || $precision -eq 12 ]]; then
continue
fi
fi
for progarg in "" "-p"; do
if [[ "$progarg" = "-p" && "$optarg" = "-o" ]]; then
continue
fi
for restartarg in "" "-r 1 -icc $IMGDIR/test3.icc" \
"-r 1b"; do
if [[ "$restartarg" = "-r 1b" && "$croparg" != "" ]]; then
continue
fi
for trimarg in "" "-tri"; do
if [ "$trimarg" = "-tri" ]; then
if [[ "$xformarg" = "-t" || "$xformarg" = "" ]]; then
continue
fi
if [ "$croparg" != "" ]; then
continue
fi
fi
runme $TJTRAN $ariarg $copyarg $croparg $xformarg \
$grayarg $optarg $progarg $restartarg $trimarg \
$OUTDIR/${basename}-$subsamp.jpg \
$OUTDIR/${basename}-tjtran.jpg
runme $EXEDIR/jpegtran $ariarg $copyarg $croparg \
$xformarg $grayarg $optarg $progarg $restartarg \
$trimarg -outf $OUTDIR/${basename}-jpegtran.jpg \
$OUTDIR/${basename}-$subsamp.jpg
$EXEDIR/test/md5sum $OUTDIR/${basename}-tjtran.jpg \
$OUTDIR/${basename}-jpegtran.jpg |
sed "s@$OUTDIR@\$OUTDIR@g"
cmp $OUTDIR/${basename}-tjtran.jpg \
$OUTDIR/${basename}-jpegtran.jpg
rm $OUTDIR/${basename}-tjtran.jpg \
$OUTDIR/${basename}-jpegtran.jpg
echo
done
done
done
done
done
done
done
done
done
rm $OUTDIR/${basename}-$subsamp.jpg
done
done
echo "GREAT SUCCESS!"
lib/libturbojpeg.a
Executable
BIN
View File
Binary file not shown.
lib/minhook/include/MinHook.h
Executable
+185
View File
@@ -0,0 +1,185 @@
/*
* MinHook - The Minimalistic API Hooking Library for x64/x86
* Copyright (C) 2009-2017 Tsuda Kageyu.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER
* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#pragma once
#if !(defined _M_IX86) && !(defined _M_X64) && !(defined __i386__) && !(defined __x86_64__)
#error MinHook supports only x86 and x64 systems.
#endif
#include <windows.h>
// MinHook Error Codes.
typedef enum MH_STATUS
{
// Unknown error. Should not be returned.
MH_UNKNOWN = -1,
// Successful.
MH_OK = 0,
// MinHook is already initialized.
MH_ERROR_ALREADY_INITIALIZED,
// MinHook is not initialized yet, or already uninitialized.
MH_ERROR_NOT_INITIALIZED,
// The hook for the specified target function is already created.
MH_ERROR_ALREADY_CREATED,
// The hook for the specified target function is not created yet.
MH_ERROR_NOT_CREATED,
// The hook for the specified target function is already enabled.
MH_ERROR_ENABLED,
// The hook for the specified target function is not enabled yet, or already
// disabled.
MH_ERROR_DISABLED,
// The specified pointer is invalid. It points the address of non-allocated
// and/or non-executable region.
MH_ERROR_NOT_EXECUTABLE,
// The specified target function cannot be hooked.
MH_ERROR_UNSUPPORTED_FUNCTION,
// Failed to allocate memory.
MH_ERROR_MEMORY_ALLOC,
// Failed to change the memory protection.
MH_ERROR_MEMORY_PROTECT,
// The specified module is not loaded.
MH_ERROR_MODULE_NOT_FOUND,
// The specified function is not found.
MH_ERROR_FUNCTION_NOT_FOUND
}
MH_STATUS;
// Can be passed as a parameter to MH_EnableHook, MH_DisableHook,
// MH_QueueEnableHook or MH_QueueDisableHook.
#define MH_ALL_HOOKS NULL
#ifdef __cplusplus
extern "C" {
#endif
// Initialize the MinHook library. You must call this function EXACTLY ONCE
// at the beginning of your program.
MH_STATUS WINAPI MH_Initialize(VOID);
// Uninitialize the MinHook library. You must call this function EXACTLY
// ONCE at the end of your program.
MH_STATUS WINAPI MH_Uninitialize(VOID);
// Creates a hook for the specified target function, in disabled state.
// Parameters:
// pTarget [in] A pointer to the target function, which will be
// overridden by the detour function.
// pDetour [in] A pointer to the detour function, which will override
// the target function.
// ppOriginal [out] A pointer to the trampoline function, which will be
// used to call the original target function.
// This parameter can be NULL.
MH_STATUS WINAPI MH_CreateHook(LPVOID pTarget, LPVOID pDetour, LPVOID *ppOriginal);
// Creates a hook for the specified API function, in disabled state.
// Parameters:
// pszModule [in] A pointer to the loaded module name which contains the
// target function.
// pszProcName [in] A pointer to the target function name, which will be
// overridden by the detour function.
// pDetour [in] A pointer to the detour function, which will override
// the target function.
// ppOriginal [out] A pointer to the trampoline function, which will be
// used to call the original target function.
// This parameter can be NULL.
MH_STATUS WINAPI MH_CreateHookApi(
LPCWSTR pszModule, LPCSTR pszProcName, LPVOID pDetour, LPVOID *ppOriginal);
// Creates a hook for the specified API function, in disabled state.
// Parameters:
// pszModule [in] A pointer to the loaded module name which contains the
// target function.
// pszProcName [in] A pointer to the target function name, which will be
// overridden by the detour function.
// pDetour [in] A pointer to the detour function, which will override
// the target function.
// ppOriginal [out] A pointer to the trampoline function, which will be
// used to call the original target function.
// This parameter can be NULL.
// ppTarget [out] A pointer to the target function, which will be used
// with other functions.
// This parameter can be NULL.
MH_STATUS WINAPI MH_CreateHookApiEx(
LPCWSTR pszModule, LPCSTR pszProcName, LPVOID pDetour, LPVOID *ppOriginal, LPVOID *ppTarget);
// Removes an already created hook.
// Parameters:
// pTarget [in] A pointer to the target function.
MH_STATUS WINAPI MH_RemoveHook(LPVOID pTarget);
// Enables an already created hook.
// Parameters:
// pTarget [in] A pointer to the target function.
// If this parameter is MH_ALL_HOOKS, all created hooks are
// enabled in one go.
MH_STATUS WINAPI MH_EnableHook(LPVOID pTarget);
// Disables an already created hook.
// Parameters:
// pTarget [in] A pointer to the target function.
// If this parameter is MH_ALL_HOOKS, all created hooks are
// disabled in one go.
MH_STATUS WINAPI MH_DisableHook(LPVOID pTarget);
// Queues to enable an already created hook.
// Parameters:
// pTarget [in] A pointer to the target function.
// If this parameter is MH_ALL_HOOKS, all created hooks are
// queued to be enabled.
MH_STATUS WINAPI MH_QueueEnableHook(LPVOID pTarget);
// Queues to disable an already created hook.
// Parameters:
// pTarget [in] A pointer to the target function.
// If this parameter is MH_ALL_HOOKS, all created hooks are
// queued to be disabled.
MH_STATUS WINAPI MH_QueueDisableHook(LPVOID pTarget);
// Applies all queued changes in one go.
MH_STATUS WINAPI MH_ApplyQueued(VOID);
// Translates the MH_STATUS to its name as a string.
const char *WINAPI MH_StatusToString(MH_STATUS status);
#ifdef __cplusplus
}
#endif
lib/minhook/src/buffer.c
Executable
+312
View File
@@ -0,0 +1,312 @@
/*
* MinHook - The Minimalistic API Hooking Library for x64/x86
* Copyright (C) 2009-2017 Tsuda Kageyu.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER
* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <windows.h>
#include "buffer.h"
// Size of each memory block. (= page size of VirtualAlloc)
#define MEMORY_BLOCK_SIZE 0x1000
// Max range for seeking a memory block. (= 1024MB)
#define MAX_MEMORY_RANGE 0x40000000
// Memory protection flags to check the executable address.
#define PAGE_EXECUTE_FLAGS \
(PAGE_EXECUTE | PAGE_EXECUTE_READ | PAGE_EXECUTE_READWRITE | PAGE_EXECUTE_WRITECOPY)
// Memory slot.
typedef struct _MEMORY_SLOT
{
union
{
struct _MEMORY_SLOT *pNext;
UINT8 buffer[MEMORY_SLOT_SIZE];
};
} MEMORY_SLOT, *PMEMORY_SLOT;
// Memory block info. Placed at the head of each block.
typedef struct _MEMORY_BLOCK
{
struct _MEMORY_BLOCK *pNext;
PMEMORY_SLOT pFree; // First element of the free slot list.
UINT usedCount;
} MEMORY_BLOCK, *PMEMORY_BLOCK;
//-------------------------------------------------------------------------
// Global Variables:
//-------------------------------------------------------------------------
// First element of the memory block list.
static PMEMORY_BLOCK g_pMemoryBlocks;
//-------------------------------------------------------------------------
VOID InitializeBuffer(VOID)
{
// Nothing to do for now.
}
//-------------------------------------------------------------------------
VOID UninitializeBuffer(VOID)
{
PMEMORY_BLOCK pBlock = g_pMemoryBlocks;
g_pMemoryBlocks = NULL;
while (pBlock)
{
PMEMORY_BLOCK pNext = pBlock->pNext;
VirtualFree(pBlock, 0, MEM_RELEASE);
pBlock = pNext;
}
}
//-------------------------------------------------------------------------
#if defined(_M_X64) || defined(__x86_64__)
static LPVOID FindPrevFreeRegion(LPVOID pAddress, LPVOID pMinAddr, DWORD dwAllocationGranularity)
{
ULONG_PTR tryAddr = (ULONG_PTR)pAddress;
// Round down to the allocation granularity.
tryAddr -= tryAddr % dwAllocationGranularity;
// Start from the previous allocation granularity multiply.
tryAddr -= dwAllocationGranularity;
while (tryAddr >= (ULONG_PTR)pMinAddr)
{
MEMORY_BASIC_INFORMATION mbi;
if (VirtualQuery((LPVOID)tryAddr, &mbi, sizeof(mbi)) == 0)
break;
if (mbi.State == MEM_FREE)
return (LPVOID)tryAddr;
if ((ULONG_PTR)mbi.AllocationBase < dwAllocationGranularity)
break;
tryAddr = (ULONG_PTR)mbi.AllocationBase - dwAllocationGranularity;
}
return NULL;
}
#endif
//-------------------------------------------------------------------------
#if defined(_M_X64) || defined(__x86_64__)
static LPVOID FindNextFreeRegion(LPVOID pAddress, LPVOID pMaxAddr, DWORD dwAllocationGranularity)
{
ULONG_PTR tryAddr = (ULONG_PTR)pAddress;
// Round down to the allocation granularity.
tryAddr -= tryAddr % dwAllocationGranularity;
// Start from the next allocation granularity multiply.
tryAddr += dwAllocationGranularity;
while (tryAddr <= (ULONG_PTR)pMaxAddr)
{
MEMORY_BASIC_INFORMATION mbi;
if (VirtualQuery((LPVOID)tryAddr, &mbi, sizeof(mbi)) == 0)
break;
if (mbi.State == MEM_FREE)
return (LPVOID)tryAddr;
tryAddr = (ULONG_PTR)mbi.BaseAddress + mbi.RegionSize;
// Round up to the next allocation granularity.
tryAddr += dwAllocationGranularity - 1;
tryAddr -= tryAddr % dwAllocationGranularity;
}
return NULL;
}
#endif
//-------------------------------------------------------------------------
static PMEMORY_BLOCK GetMemoryBlock(LPVOID pOrigin)
{
PMEMORY_BLOCK pBlock;
#if defined(_M_X64) || defined(__x86_64__)
ULONG_PTR minAddr;
ULONG_PTR maxAddr;
SYSTEM_INFO si;
GetSystemInfo(&si);
minAddr = (ULONG_PTR)si.lpMinimumApplicationAddress;
maxAddr = (ULONG_PTR)si.lpMaximumApplicationAddress;
// pOrigin ± 512MB
if ((ULONG_PTR)pOrigin > MAX_MEMORY_RANGE && minAddr < (ULONG_PTR)pOrigin - MAX_MEMORY_RANGE)
minAddr = (ULONG_PTR)pOrigin - MAX_MEMORY_RANGE;
if (maxAddr > (ULONG_PTR)pOrigin + MAX_MEMORY_RANGE)
maxAddr = (ULONG_PTR)pOrigin + MAX_MEMORY_RANGE;
// Make room for MEMORY_BLOCK_SIZE bytes.
maxAddr -= MEMORY_BLOCK_SIZE - 1;
#endif
// Look the registered blocks for a reachable one.
for (pBlock = g_pMemoryBlocks; pBlock != NULL; pBlock = pBlock->pNext)
{
#if defined(_M_X64) || defined(__x86_64__)
// Ignore the blocks too far.
if ((ULONG_PTR)pBlock < minAddr || (ULONG_PTR)pBlock >= maxAddr)
continue;
#endif
// The block has at least one unused slot.
if (pBlock->pFree != NULL)
return pBlock;
}
#if defined(_M_X64) || defined(__x86_64__)
// Alloc a new block above if not found.
{
LPVOID pAlloc = pOrigin;
while ((ULONG_PTR)pAlloc >= minAddr)
{
pAlloc = FindPrevFreeRegion(pAlloc, (LPVOID)minAddr, si.dwAllocationGranularity);
if (pAlloc == NULL)
break;
pBlock = (PMEMORY_BLOCK)VirtualAlloc(
pAlloc, MEMORY_BLOCK_SIZE, MEM_COMMIT | MEM_RESERVE, PAGE_EXECUTE_READWRITE);
if (pBlock != NULL)
break;
}
}
// Alloc a new block below if not found.
if (pBlock == NULL)
{
LPVOID pAlloc = pOrigin;
while ((ULONG_PTR)pAlloc <= maxAddr)
{
pAlloc = FindNextFreeRegion(pAlloc, (LPVOID)maxAddr, si.dwAllocationGranularity);
if (pAlloc == NULL)
break;
pBlock = (PMEMORY_BLOCK)VirtualAlloc(
pAlloc, MEMORY_BLOCK_SIZE, MEM_COMMIT | MEM_RESERVE, PAGE_EXECUTE_READWRITE);
if (pBlock != NULL)
break;
}
}
#else
// In x86 mode, a memory block can be placed anywhere.
pBlock = (PMEMORY_BLOCK)VirtualAlloc(
NULL, MEMORY_BLOCK_SIZE, MEM_COMMIT | MEM_RESERVE, PAGE_EXECUTE_READWRITE);
#endif
if (pBlock != NULL)
{
// Build a linked list of all the slots.
PMEMORY_SLOT pSlot = (PMEMORY_SLOT)pBlock + 1;
pBlock->pFree = NULL;
pBlock->usedCount = 0;
do
{
pSlot->pNext = pBlock->pFree;
pBlock->pFree = pSlot;
pSlot++;
} while ((ULONG_PTR)pSlot - (ULONG_PTR)pBlock <= MEMORY_BLOCK_SIZE - MEMORY_SLOT_SIZE);
pBlock->pNext = g_pMemoryBlocks;
g_pMemoryBlocks = pBlock;
}
return pBlock;
}
//-------------------------------------------------------------------------
LPVOID AllocateBuffer(LPVOID pOrigin)
{
PMEMORY_SLOT pSlot;
PMEMORY_BLOCK pBlock = GetMemoryBlock(pOrigin);
if (pBlock == NULL)
return NULL;
// Remove an unused slot from the list.
pSlot = pBlock->pFree;
pBlock->pFree = pSlot->pNext;
pBlock->usedCount++;
#ifdef _DEBUG
// Fill the slot with INT3 for debugging.
memset(pSlot, 0xCC, sizeof(MEMORY_SLOT));
#endif
return pSlot;
}
//-------------------------------------------------------------------------
VOID FreeBuffer(LPVOID pBuffer)
{
PMEMORY_BLOCK pBlock = g_pMemoryBlocks;
PMEMORY_BLOCK pPrev = NULL;
ULONG_PTR pTargetBlock = ((ULONG_PTR)pBuffer / MEMORY_BLOCK_SIZE) * MEMORY_BLOCK_SIZE;
while (pBlock != NULL)
{
if ((ULONG_PTR)pBlock == pTargetBlock)
{
PMEMORY_SLOT pSlot = (PMEMORY_SLOT)pBuffer;
#ifdef _DEBUG
// Clear the released slot for debugging.
memset(pSlot, 0x00, sizeof(MEMORY_SLOT));
#endif
// Restore the released slot to the list.
pSlot->pNext = pBlock->pFree;
pBlock->pFree = pSlot;
pBlock->usedCount--;
// Free if unused.
if (pBlock->usedCount == 0)
{
if (pPrev)
pPrev->pNext = pBlock->pNext;
else
g_pMemoryBlocks = pBlock->pNext;
VirtualFree(pBlock, 0, MEM_RELEASE);
}
break;
}
pPrev = pBlock;
pBlock = pBlock->pNext;
}
}
//-------------------------------------------------------------------------
BOOL IsExecutableAddress(LPVOID pAddress)
{
MEMORY_BASIC_INFORMATION mi;
VirtualQuery(pAddress, &mi, sizeof(mi));
return (mi.State == MEM_COMMIT && (mi.Protect & PAGE_EXECUTE_FLAGS));
}
lib/minhook/src/buffer.h
Executable
+42
View File
@@ -0,0 +1,42 @@
/*
* MinHook - The Minimalistic API Hooking Library for x64/x86
* Copyright (C) 2009-2017 Tsuda Kageyu.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER
* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#pragma once
// Size of each memory slot.
#if defined(_M_X64) || defined(__x86_64__)
#define MEMORY_SLOT_SIZE 64
#else
#define MEMORY_SLOT_SIZE 32
#endif
VOID InitializeBuffer(VOID);
VOID UninitializeBuffer(VOID);
LPVOID AllocateBuffer(LPVOID pOrigin);
VOID FreeBuffer(LPVOID pBuffer);
BOOL IsExecutableAddress(LPVOID pAddress);
lib/minhook/src/hde/hde32.c
Executable
+324
View File
@@ -0,0 +1,324 @@
/*
* Hacker Disassembler Engine 32 C
* Copyright (c) 2008-2009, Vyacheslav Patkov.
* All rights reserved.
*
*/
#if defined(_M_IX86) || defined(__i386__)
#include <string.h>
#include "hde32.h"
#include "table32.h"
unsigned int hde32_disasm(const void *code, hde32s *hs)
{
uint8_t x, c, *p = (uint8_t *)code, cflags, opcode, pref = 0;
uint8_t *ht = hde32_table, m_mod, m_reg, m_rm, disp_size = 0;
memset(hs, 0, sizeof(hde32s));
for (x = 16; x; x--)
switch (c = *p++) {
case 0xf3:
hs->p_rep = c;
pref |= PRE_F3;
break;
case 0xf2:
hs->p_rep = c;
pref |= PRE_F2;
break;
case 0xf0:
hs->p_lock = c;
pref |= PRE_LOCK;
break;
case 0x26: case 0x2e: case 0x36:
case 0x3e: case 0x64: case 0x65:
hs->p_seg = c;
pref |= PRE_SEG;
break;
case 0x66:
hs->p_66 = c;
pref |= PRE_66;
break;
case 0x67:
hs->p_67 = c;
pref |= PRE_67;
break;
default:
goto pref_done;
}
pref_done:
hs->flags = (uint32_t)pref << 23;
if (!pref)
pref |= PRE_NONE;
if ((hs->opcode = c) == 0x0f) {
hs->opcode2 = c = *p++;
ht += DELTA_OPCODES;
} else if (c >= 0xa0 && c <= 0xa3) {
if (pref & PRE_67)
pref |= PRE_66;
else
pref &= ~PRE_66;
}
opcode = c;
cflags = ht[ht[opcode / 4] + (opcode % 4)];
if (cflags == C_ERROR) {
hs->flags |= F_ERROR | F_ERROR_OPCODE;
cflags = 0;
if ((opcode & -3) == 0x24)
cflags++;
}
x = 0;
if (cflags & C_GROUP) {
uint16_t t;
t = *(uint16_t *)(ht + (cflags & 0x7f));
cflags = (uint8_t)t;
x = (uint8_t)(t >> 8);
}
if (hs->opcode2) {
ht = hde32_table + DELTA_PREFIXES;
if (ht[ht[opcode / 4] + (opcode % 4)] & pref)
hs->flags |= F_ERROR | F_ERROR_OPCODE;
}
if (cflags & C_MODRM) {
hs->flags |= F_MODRM;
hs->modrm = c = *p++;
hs->modrm_mod = m_mod = c >> 6;
hs->modrm_rm = m_rm = c & 7;
hs->modrm_reg = m_reg = (c & 0x3f) >> 3;
if (x && ((x << m_reg) & 0x80))
hs->flags |= F_ERROR | F_ERROR_OPCODE;
if (!hs->opcode2 && opcode >= 0xd9 && opcode <= 0xdf) {
uint8_t t = opcode - 0xd9;
if (m_mod == 3) {
ht = hde32_table + DELTA_FPU_MODRM + t*8;
t = ht[m_reg] << m_rm;
} else {
ht = hde32_table + DELTA_FPU_REG;
t = ht[t] << m_reg;
}
if (t & 0x80)
hs->flags |= F_ERROR | F_ERROR_OPCODE;
}
if (pref & PRE_LOCK) {
if (m_mod == 3) {
hs->flags |= F_ERROR | F_ERROR_LOCK;
} else {
uint8_t *table_end, op = opcode;
if (hs->opcode2) {
ht = hde32_table + DELTA_OP2_LOCK_OK;
table_end = ht + DELTA_OP_ONLY_MEM - DELTA_OP2_LOCK_OK;
} else {
ht = hde32_table + DELTA_OP_LOCK_OK;
table_end = ht + DELTA_OP2_LOCK_OK - DELTA_OP_LOCK_OK;
op &= -2;
}
for (; ht != table_end; ht++)
if (*ht++ == op) {
if (!((*ht << m_reg) & 0x80))
goto no_lock_error;
else
break;
}
hs->flags |= F_ERROR | F_ERROR_LOCK;
no_lock_error:
;
}
}
if (hs->opcode2) {
switch (opcode) {
case 0x20: case 0x22:
m_mod = 3;
if (m_reg > 4 || m_reg == 1)
goto error_operand;
else
goto no_error_operand;
case 0x21: case 0x23:
m_mod = 3;
if (m_reg == 4 || m_reg == 5)
goto error_operand;
else
goto no_error_operand;
}
} else {
switch (opcode) {
case 0x8c:
if (m_reg > 5)
goto error_operand;
else
goto no_error_operand;
case 0x8e:
if (m_reg == 1 || m_reg > 5)
goto error_operand;
else
goto no_error_operand;
}
}
if (m_mod == 3) {
uint8_t *table_end;
if (hs->opcode2) {
ht = hde32_table + DELTA_OP2_ONLY_MEM;
table_end = ht + sizeof(hde32_table) - DELTA_OP2_ONLY_MEM;
} else {
ht = hde32_table + DELTA_OP_ONLY_MEM;
table_end = ht + DELTA_OP2_ONLY_MEM - DELTA_OP_ONLY_MEM;
}
for (; ht != table_end; ht += 2)
if (*ht++ == opcode) {
if ((*ht++ & pref) && !((*ht << m_reg) & 0x80))
goto error_operand;
else
break;
}
goto no_error_operand;
} else if (hs->opcode2) {
switch (opcode) {
case 0x50: case 0xd7: case 0xf7:
if (pref & (PRE_NONE | PRE_66))
goto error_operand;
break;
case 0xd6:
if (pref & (PRE_F2 | PRE_F3))
goto error_operand;
break;
case 0xc5:
goto error_operand;
}
goto no_error_operand;
} else
goto no_error_operand;
error_operand:
hs->flags |= F_ERROR | F_ERROR_OPERAND;
no_error_operand:
c = *p++;
if (m_reg <= 1) {
if (opcode == 0xf6)
cflags |= C_IMM8;
else if (opcode == 0xf7)
cflags |= C_IMM_P66;
}
switch (m_mod) {
case 0:
if (pref & PRE_67) {
if (m_rm == 6)
disp_size = 2;
} else
if (m_rm == 5)
disp_size = 4;
break;
case 1:
disp_size = 1;
break;
case 2:
disp_size = 2;
if (!(pref & PRE_67))
disp_size <<= 1;
break;
}
if (m_mod != 3 && m_rm == 4 && !(pref & PRE_67)) {
hs->flags |= F_SIB;
p++;
hs->sib = c;
hs->sib_scale = c >> 6;
hs->sib_index = (c & 0x3f) >> 3;
if ((hs->sib_base = c & 7) == 5 && !(m_mod & 1))
disp_size = 4;
}
p--;
switch (disp_size) {
case 1:
hs->flags |= F_DISP8;
hs->disp.disp8 = *p;
break;
case 2:
hs->flags |= F_DISP16;
hs->disp.disp16 = *(uint16_t *)p;
break;
case 4:
hs->flags |= F_DISP32;
hs->disp.disp32 = *(uint32_t *)p;
break;
}
p += disp_size;
} else if (pref & PRE_LOCK)
hs->flags |= F_ERROR | F_ERROR_LOCK;
if (cflags & C_IMM_P66) {
if (cflags & C_REL32) {
if (pref & PRE_66) {
hs->flags |= F_IMM16 | F_RELATIVE;
hs->imm.imm16 = *(uint16_t *)p;
p += 2;
goto disasm_done;
}
goto rel32_ok;
}
if (pref & PRE_66) {
hs->flags |= F_IMM16;
hs->imm.imm16 = *(uint16_t *)p;
p += 2;
} else {
hs->flags |= F_IMM32;
hs->imm.imm32 = *(uint32_t *)p;
p += 4;
}
}
if (cflags & C_IMM16) {
if (hs->flags & F_IMM32) {
hs->flags |= F_IMM16;
hs->disp.disp16 = *(uint16_t *)p;
} else if (hs->flags & F_IMM16) {
hs->flags |= F_2IMM16;
hs->disp.disp16 = *(uint16_t *)p;
} else {
hs->flags |= F_IMM16;
hs->imm.imm16 = *(uint16_t *)p;
}
p += 2;
}
if (cflags & C_IMM8) {
hs->flags |= F_IMM8;
hs->imm.imm8 = *p++;
}
if (cflags & C_REL32) {
rel32_ok:
hs->flags |= F_IMM32 | F_RELATIVE;
hs->imm.imm32 = *(uint32_t *)p;
p += 4;
} else if (cflags & C_REL8) {
hs->flags |= F_IMM8 | F_RELATIVE;
hs->imm.imm8 = *p++;
}
disasm_done:
if ((hs->len = (uint8_t)(p-(uint8_t *)code)) > 15) {
hs->flags |= F_ERROR | F_ERROR_LENGTH;
hs->len = 15;
}
return (unsigned int)hs->len;
}
#endif // defined(_M_IX86) || defined(__i386__)
lib/minhook/src/hde/hde32.h
Executable
+105
View File
@@ -0,0 +1,105 @@
/*
* Hacker Disassembler Engine 32
* Copyright (c) 2006-2009, Vyacheslav Patkov.
* All rights reserved.
*
* hde32.h: C/C++ header file
*
*/
#ifndef _HDE32_H_
#define _HDE32_H_
/* stdint.h - C99 standard header
* http://en.wikipedia.org/wiki/stdint.h
*
* if your compiler doesn't contain "stdint.h" header (for
* example, Microsoft Visual C++), you can download file:
* http://www.azillionmonkeys.com/qed/pstdint.h
* and change next line to:
* #include "pstdint.h"
*/
#include "pstdint.h"
#define F_MODRM 0x00000001
#define F_SIB 0x00000002
#define F_IMM8 0x00000004
#define F_IMM16 0x00000008
#define F_IMM32 0x00000010
#define F_DISP8 0x00000020
#define F_DISP16 0x00000040
#define F_DISP32 0x00000080
#define F_RELATIVE 0x00000100
#define F_2IMM16 0x00000800
#define F_ERROR 0x00001000
#define F_ERROR_OPCODE 0x00002000
#define F_ERROR_LENGTH 0x00004000
#define F_ERROR_LOCK 0x00008000
#define F_ERROR_OPERAND 0x00010000
#define F_PREFIX_REPNZ 0x01000000
#define F_PREFIX_REPX 0x02000000
#define F_PREFIX_REP 0x03000000
#define F_PREFIX_66 0x04000000
#define F_PREFIX_67 0x08000000
#define F_PREFIX_LOCK 0x10000000
#define F_PREFIX_SEG 0x20000000
#define F_PREFIX_ANY 0x3f000000
#define PREFIX_SEGMENT_CS 0x2e
#define PREFIX_SEGMENT_SS 0x36
#define PREFIX_SEGMENT_DS 0x3e
#define PREFIX_SEGMENT_ES 0x26
#define PREFIX_SEGMENT_FS 0x64
#define PREFIX_SEGMENT_GS 0x65
#define PREFIX_LOCK 0xf0
#define PREFIX_REPNZ 0xf2
#define PREFIX_REPX 0xf3
#define PREFIX_OPERAND_SIZE 0x66
#define PREFIX_ADDRESS_SIZE 0x67
#pragma pack(push,1)
typedef struct {
uint8_t len;
uint8_t p_rep;
uint8_t p_lock;
uint8_t p_seg;
uint8_t p_66;
uint8_t p_67;
uint8_t opcode;
uint8_t opcode2;
uint8_t modrm;
uint8_t modrm_mod;
uint8_t modrm_reg;
uint8_t modrm_rm;
uint8_t sib;
uint8_t sib_scale;
uint8_t sib_index;
uint8_t sib_base;
union {
uint8_t imm8;
uint16_t imm16;
uint32_t imm32;
} imm;
union {
uint8_t disp8;
uint16_t disp16;
uint32_t disp32;
} disp;
uint32_t flags;
} hde32s;
#pragma pack(pop)
#ifdef __cplusplus
extern "C" {
#endif
/* __cdecl */
unsigned int hde32_disasm(const void *code, hde32s *hs);
#ifdef __cplusplus
}
#endif
#endif /* _HDE32_H_ */
lib/minhook/src/hde/hde64.c
Executable
+335
View File
@@ -0,0 +1,335 @@
/*
* Hacker Disassembler Engine 64 C
* Copyright (c) 2008-2009, Vyacheslav Patkov.
* All rights reserved.
*
*/
#if defined(_M_X64) || defined(__x86_64__)
#include <string.h>
#include "hde64.h"
#include "table64.h"
unsigned int hde64_disasm(const void *code, hde64s *hs)
{
uint8_t x, c, *p = (uint8_t *)code, cflags, opcode, pref = 0;
uint8_t *ht = hde64_table, m_mod, m_reg, m_rm, disp_size = 0;
uint8_t op64 = 0;
memset(hs, 0, sizeof(hde64s));
for (x = 16; x; x--)
switch (c = *p++) {
case 0xf3:
hs->p_rep = c;
pref |= PRE_F3;
break;
case 0xf2:
hs->p_rep = c;
pref |= PRE_F2;
break;
case 0xf0:
hs->p_lock = c;
pref |= PRE_LOCK;
break;
case 0x26: case 0x2e: case 0x36:
case 0x3e: case 0x64: case 0x65:
hs->p_seg = c;
pref |= PRE_SEG;
break;
case 0x66:
hs->p_66 = c;
pref |= PRE_66;
break;
case 0x67:
hs->p_67 = c;
pref |= PRE_67;
break;
default:
goto pref_done;
}
pref_done:
hs->flags = (uint32_t)pref << 23;
if (!pref)
pref |= PRE_NONE;
if ((c & 0xf0) == 0x40) {
hs->flags |= F_PREFIX_REX;
if ((hs->rex_w = (c & 0xf) >> 3) && (*p & 0xf8) == 0xb8)
op64++;
hs->rex_r = (c & 7) >> 2;
hs->rex_x = (c & 3) >> 1;
hs->rex_b = c & 1;
if (((c = *p++) & 0xf0) == 0x40) {
opcode = c;
goto error_opcode;
}
}
if ((hs->opcode = c) == 0x0f) {
hs->opcode2 = c = *p++;
ht += DELTA_OPCODES;
} else if (c >= 0xa0 && c <= 0xa3) {
op64++;
if (pref & PRE_67)
pref |= PRE_66;
else
pref &= ~PRE_66;
}
opcode = c;
cflags = ht[ht[opcode / 4] + (opcode % 4)];
if (cflags == C_ERROR) {
error_opcode:
hs->flags |= F_ERROR | F_ERROR_OPCODE;
cflags = 0;
if ((opcode & -3) == 0x24)
cflags++;
}
x = 0;
if (cflags & C_GROUP) {
uint16_t t;
t = *(uint16_t *)(ht + (cflags & 0x7f));
cflags = (uint8_t)t;
x = (uint8_t)(t >> 8);
}
if (hs->opcode2) {
ht = hde64_table + DELTA_PREFIXES;
if (ht[ht[opcode / 4] + (opcode % 4)] & pref)
hs->flags |= F_ERROR | F_ERROR_OPCODE;
}
if (cflags & C_MODRM) {
hs->flags |= F_MODRM;
hs->modrm = c = *p++;
hs->modrm_mod = m_mod = c >> 6;
hs->modrm_rm = m_rm = c & 7;
hs->modrm_reg = m_reg = (c & 0x3f) >> 3;
if (x && ((x << m_reg) & 0x80))
hs->flags |= F_ERROR | F_ERROR_OPCODE;
if (!hs->opcode2 && opcode >= 0xd9 && opcode <= 0xdf) {
uint8_t t = opcode - 0xd9;
if (m_mod == 3) {
ht = hde64_table + DELTA_FPU_MODRM + t*8;
t = ht[m_reg] << m_rm;
} else {
ht = hde64_table + DELTA_FPU_REG;
t = ht[t] << m_reg;
}
if (t & 0x80)
hs->flags |= F_ERROR | F_ERROR_OPCODE;
}
if (pref & PRE_LOCK) {
if (m_mod == 3) {
hs->flags |= F_ERROR | F_ERROR_LOCK;
} else {
uint8_t *table_end, op = opcode;
if (hs->opcode2) {
ht = hde64_table + DELTA_OP2_LOCK_OK;
table_end = ht + DELTA_OP_ONLY_MEM - DELTA_OP2_LOCK_OK;
} else {
ht = hde64_table + DELTA_OP_LOCK_OK;
table_end = ht + DELTA_OP2_LOCK_OK - DELTA_OP_LOCK_OK;
op &= -2;
}
for (; ht != table_end; ht++)
if (*ht++ == op) {
if (!((*ht << m_reg) & 0x80))
goto no_lock_error;
else
break;
}
hs->flags |= F_ERROR | F_ERROR_LOCK;
no_lock_error:
;
}
}
if (hs->opcode2) {
switch (opcode) {
case 0x20: case 0x22:
m_mod = 3;
if (m_reg > 4 || m_reg == 1)
goto error_operand;
else
goto no_error_operand;
case 0x21: case 0x23:
m_mod = 3;
if (m_reg == 4 || m_reg == 5)
goto error_operand;
else
goto no_error_operand;
}
} else {
switch (opcode) {
case 0x8c:
if (m_reg > 5)
goto error_operand;
else
goto no_error_operand;
case 0x8e:
if (m_reg == 1 || m_reg > 5)
goto error_operand;
else
goto no_error_operand;
}
}
if (m_mod == 3) {
uint8_t *table_end;
if (hs->opcode2) {
ht = hde64_table + DELTA_OP2_ONLY_MEM;
table_end = ht + sizeof(hde64_table) - DELTA_OP2_ONLY_MEM;
} else {
ht = hde64_table + DELTA_OP_ONLY_MEM;
table_end = ht + DELTA_OP2_ONLY_MEM - DELTA_OP_ONLY_MEM;
}
for (; ht != table_end; ht += 2)
if (*ht++ == opcode) {
if ((*ht++ & pref) && !((*ht << m_reg) & 0x80))
goto error_operand;
else
break;
}
goto no_error_operand;
} else if (hs->opcode2) {
switch (opcode) {
case 0x50: case 0xd7: case 0xf7:
if (pref & (PRE_NONE | PRE_66))
goto error_operand;
break;
case 0xd6:
if (pref & (PRE_F2 | PRE_F3))
goto error_operand;
break;
case 0xc5:
goto error_operand;
}
goto no_error_operand;
} else
goto no_error_operand;
error_operand:
hs->flags |= F_ERROR | F_ERROR_OPERAND;
no_error_operand:
c = *p++;
if (m_reg <= 1) {
if (opcode == 0xf6)
cflags |= C_IMM8;
else if (opcode == 0xf7)
cflags |= C_IMM_P66;
}
switch (m_mod) {
case 0:
if (pref & PRE_67) {
if (m_rm == 6)
disp_size = 2;
} else
if (m_rm == 5)
disp_size = 4;
break;
case 1:
disp_size = 1;
break;
case 2:
disp_size = 2;
if (!(pref & PRE_67))
disp_size <<= 1;
break;
}
if (m_mod != 3 && m_rm == 4) {
hs->flags |= F_SIB;
p++;
hs->sib = c;
hs->sib_scale = c >> 6;
hs->sib_index = (c & 0x3f) >> 3;
if ((hs->sib_base = c & 7) == 5 && !(m_mod & 1))
disp_size = 4;
}
p--;
switch (disp_size) {
case 1:
hs->flags |= F_DISP8;
hs->disp.disp8 = *p;
break;
case 2:
hs->flags |= F_DISP16;
hs->disp.disp16 = *(uint16_t *)p;
break;
case 4:
hs->flags |= F_DISP32;
hs->disp.disp32 = *(uint32_t *)p;
break;
}
p += disp_size;
} else if (pref & PRE_LOCK)
hs->flags |= F_ERROR | F_ERROR_LOCK;
if (cflags & C_IMM_P66) {
if (cflags & C_REL32) {
if (pref & PRE_66) {
hs->flags |= F_IMM16 | F_RELATIVE;
hs->imm.imm16 = *(uint16_t *)p;
p += 2;
goto disasm_done;
}
goto rel32_ok;
}
if (op64) {
hs->flags |= F_IMM64;
hs->imm.imm64 = *(uint64_t *)p;
p += 8;
} else if (!(pref & PRE_66)) {
hs->flags |= F_IMM32;
hs->imm.imm32 = *(uint32_t *)p;
p += 4;
} else
goto imm16_ok;
}
if (cflags & C_IMM16) {
imm16_ok:
hs->flags |= F_IMM16;
hs->imm.imm16 = *(uint16_t *)p;
p += 2;
}
if (cflags & C_IMM8) {
hs->flags |= F_IMM8;
hs->imm.imm8 = *p++;
}
if (cflags & C_REL32) {
rel32_ok:
hs->flags |= F_IMM32 | F_RELATIVE;
hs->imm.imm32 = *(uint32_t *)p;
p += 4;
} else if (cflags & C_REL8) {
hs->flags |= F_IMM8 | F_RELATIVE;
hs->imm.imm8 = *p++;
}
disasm_done:
if ((hs->len = (uint8_t)(p-(uint8_t *)code)) > 15) {
hs->flags |= F_ERROR | F_ERROR_LENGTH;
hs->len = 15;
}
return (unsigned int)hs->len;
}
#endif // defined(_M_X64) || defined(__x86_64__)
lib/minhook/src/hde/hde64.h
Executable
+112
View File
@@ -0,0 +1,112 @@
/*
* Hacker Disassembler Engine 64
* Copyright (c) 2008-2009, Vyacheslav Patkov.
* All rights reserved.
*
* hde64.h: C/C++ header file
*
*/
#ifndef _HDE64_H_
#define _HDE64_H_
/* stdint.h - C99 standard header
* http://en.wikipedia.org/wiki/stdint.h
*
* if your compiler doesn't contain "stdint.h" header (for
* example, Microsoft Visual C++), you can download file:
* http://www.azillionmonkeys.com/qed/pstdint.h
* and change next line to:
* #include "pstdint.h"
*/
#include "pstdint.h"
#define F_MODRM 0x00000001
#define F_SIB 0x00000002
#define F_IMM8 0x00000004
#define F_IMM16 0x00000008
#define F_IMM32 0x00000010
#define F_IMM64 0x00000020
#define F_DISP8 0x00000040
#define F_DISP16 0x00000080
#define F_DISP32 0x00000100
#define F_RELATIVE 0x00000200
#define F_ERROR 0x00001000
#define F_ERROR_OPCODE 0x00002000
#define F_ERROR_LENGTH 0x00004000
#define F_ERROR_LOCK 0x00008000
#define F_ERROR_OPERAND 0x00010000
#define F_PREFIX_REPNZ 0x01000000
#define F_PREFIX_REPX 0x02000000
#define F_PREFIX_REP 0x03000000
#define F_PREFIX_66 0x04000000
#define F_PREFIX_67 0x08000000
#define F_PREFIX_LOCK 0x10000000
#define F_PREFIX_SEG 0x20000000
#define F_PREFIX_REX 0x40000000
#define F_PREFIX_ANY 0x7f000000
#define PREFIX_SEGMENT_CS 0x2e
#define PREFIX_SEGMENT_SS 0x36
#define PREFIX_SEGMENT_DS 0x3e
#define PREFIX_SEGMENT_ES 0x26
#define PREFIX_SEGMENT_FS 0x64
#define PREFIX_SEGMENT_GS 0x65
#define PREFIX_LOCK 0xf0
#define PREFIX_REPNZ 0xf2
#define PREFIX_REPX 0xf3
#define PREFIX_OPERAND_SIZE 0x66
#define PREFIX_ADDRESS_SIZE 0x67
#pragma pack(push,1)
typedef struct {
uint8_t len;
uint8_t p_rep;
uint8_t p_lock;
uint8_t p_seg;
uint8_t p_66;
uint8_t p_67;
uint8_t rex;
uint8_t rex_w;
uint8_t rex_r;
uint8_t rex_x;
uint8_t rex_b;
uint8_t opcode;
uint8_t opcode2;
uint8_t modrm;
uint8_t modrm_mod;
uint8_t modrm_reg;
uint8_t modrm_rm;
uint8_t sib;
uint8_t sib_scale;
uint8_t sib_index;
uint8_t sib_base;
union {
uint8_t imm8;
uint16_t imm16;
uint32_t imm32;
uint64_t imm64;
} imm;
union {
uint8_t disp8;
uint16_t disp16;
uint32_t disp32;
} disp;
uint32_t flags;
} hde64s;
#pragma pack(pop)
#ifdef __cplusplus
extern "C" {
#endif
/* __cdecl */
unsigned int hde64_disasm(const void *code, hde64s *hs);
#ifdef __cplusplus
}
#endif
#endif /* _HDE64_H_ */
lib/minhook/src/hde/pstdint.h
Executable
+39
View File
@@ -0,0 +1,39 @@
/*
* MinHook - The Minimalistic API Hooking Library for x64/x86
* Copyright (C) 2009-2017 Tsuda Kageyu. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR "AS IS" AND ANY EXPRESS OR
* IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
* IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
* NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
* THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#pragma once
#include <windows.h>
// Integer types for HDE.
typedef INT8 int8_t;
typedef INT16 int16_t;
typedef INT32 int32_t;
typedef INT64 int64_t;
typedef UINT8 uint8_t;
typedef UINT16 uint16_t;
typedef UINT32 uint32_t;
typedef UINT64 uint64_t;
lib/minhook/src/hde/table32.h
Executable
+73
View File
@@ -0,0 +1,73 @@
/*
* Hacker Disassembler Engine 32 C
* Copyright (c) 2008-2009, Vyacheslav Patkov.
* All rights reserved.
*
*/
#define C_NONE 0x00
#define C_MODRM 0x01
#define C_IMM8 0x02
#define C_IMM16 0x04
#define C_IMM_P66 0x10
#define C_REL8 0x20
#define C_REL32 0x40
#define C_GROUP 0x80
#define C_ERROR 0xff
#define PRE_ANY 0x00
#define PRE_NONE 0x01
#define PRE_F2 0x02
#define PRE_F3 0x04
#define PRE_66 0x08
#define PRE_67 0x10
#define PRE_LOCK 0x20
#define PRE_SEG 0x40
#define PRE_ALL 0xff
#define DELTA_OPCODES 0x4a
#define DELTA_FPU_REG 0xf1
#define DELTA_FPU_MODRM 0xf8
#define DELTA_PREFIXES 0x130
#define DELTA_OP_LOCK_OK 0x1a1
#define DELTA_OP2_LOCK_OK 0x1b9
#define DELTA_OP_ONLY_MEM 0x1cb
#define DELTA_OP2_ONLY_MEM 0x1da
unsigned char hde32_table[] = {
0xa3,0xa8,0xa3,0xa8,0xa3,0xa8,0xa3,0xa8,0xa3,0xa8,0xa3,0xa8,0xa3,0xa8,0xa3,
0xa8,0xaa,0xaa,0xaa,0xaa,0xaa,0xaa,0xaa,0xaa,0xac,0xaa,0xb2,0xaa,0x9f,0x9f,
0x9f,0x9f,0xb5,0xa3,0xa3,0xa4,0xaa,0xaa,0xba,0xaa,0x96,0xaa,0xa8,0xaa,0xc3,
0xc3,0x96,0x96,0xb7,0xae,0xd6,0xbd,0xa3,0xc5,0xa3,0xa3,0x9f,0xc3,0x9c,0xaa,
0xaa,0xac,0xaa,0xbf,0x03,0x7f,0x11,0x7f,0x01,0x7f,0x01,0x3f,0x01,0x01,0x90,
0x82,0x7d,0x97,0x59,0x59,0x59,0x59,0x59,0x7f,0x59,0x59,0x60,0x7d,0x7f,0x7f,
0x59,0x59,0x59,0x59,0x59,0x59,0x59,0x59,0x59,0x59,0x59,0x59,0x9a,0x88,0x7d,
0x59,0x50,0x50,0x50,0x50,0x59,0x59,0x59,0x59,0x61,0x94,0x61,0x9e,0x59,0x59,
0x85,0x59,0x92,0xa3,0x60,0x60,0x59,0x59,0x59,0x59,0x59,0x59,0x59,0x59,0x59,
0x59,0x59,0x9f,0x01,0x03,0x01,0x04,0x03,0xd5,0x03,0xcc,0x01,0xbc,0x03,0xf0,
0x10,0x10,0x10,0x10,0x50,0x50,0x50,0x50,0x14,0x20,0x20,0x20,0x20,0x01,0x01,
0x01,0x01,0xc4,0x02,0x10,0x00,0x00,0x00,0x00,0x01,0x01,0xc0,0xc2,0x10,0x11,
0x02,0x03,0x11,0x03,0x03,0x04,0x00,0x00,0x14,0x00,0x02,0x00,0x00,0xc6,0xc8,
0x02,0x02,0x02,0x02,0x00,0x00,0xff,0xff,0xff,0xff,0x00,0x00,0x00,0xff,0xca,
0x01,0x01,0x01,0x00,0x06,0x00,0x04,0x00,0xc0,0xc2,0x01,0x01,0x03,0x01,0xff,
0xff,0x01,0x00,0x03,0xc4,0xc4,0xc6,0x03,0x01,0x01,0x01,0xff,0x03,0x03,0x03,
0xc8,0x40,0x00,0x0a,0x00,0x04,0x00,0x00,0x00,0x00,0x7f,0x00,0x33,0x01,0x00,
0x00,0x00,0x00,0x00,0x00,0xff,0xbf,0xff,0xff,0x00,0x00,0x00,0x00,0x07,0x00,
0x00,0xff,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
0x00,0xff,0xff,0x00,0x00,0x00,0xbf,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
0x7f,0x00,0x00,0xff,0x4a,0x4a,0x4a,0x4a,0x4b,0x52,0x4a,0x4a,0x4a,0x4a,0x4f,
0x4c,0x4a,0x4a,0x4a,0x4a,0x4a,0x4a,0x4a,0x4a,0x55,0x45,0x40,0x4a,0x4a,0x4a,
0x45,0x59,0x4d,0x46,0x4a,0x5d,0x4a,0x4a,0x4a,0x4a,0x4a,0x4a,0x4a,0x4a,0x4a,
0x4a,0x4a,0x4a,0x4a,0x4a,0x61,0x63,0x67,0x4e,0x4a,0x4a,0x6b,0x6d,0x4a,0x4a,
0x45,0x6d,0x4a,0x4a,0x44,0x45,0x4a,0x4a,0x00,0x00,0x00,0x02,0x0d,0x06,0x06,
0x06,0x06,0x0e,0x00,0x00,0x00,0x00,0x06,0x06,0x06,0x00,0x06,0x06,0x02,0x06,
0x00,0x0a,0x0a,0x07,0x07,0x06,0x02,0x05,0x05,0x02,0x02,0x00,0x00,0x04,0x04,
0x04,0x04,0x00,0x00,0x00,0x0e,0x05,0x06,0x06,0x06,0x01,0x06,0x00,0x00,0x08,
0x00,0x10,0x00,0x18,0x00,0x20,0x00,0x28,0x00,0x30,0x00,0x80,0x01,0x82,0x01,
0x86,0x00,0xf6,0xcf,0xfe,0x3f,0xab,0x00,0xb0,0x00,0xb1,0x00,0xb3,0x00,0xba,
0xf8,0xbb,0x00,0xc0,0x00,0xc1,0x00,0xc7,0xbf,0x62,0xff,0x00,0x8d,0xff,0x00,
0xc4,0xff,0x00,0xc5,0xff,0x00,0xff,0xff,0xeb,0x01,0xff,0x0e,0x12,0x08,0x00,
0x13,0x09,0x00,0x16,0x08,0x00,0x17,0x09,0x00,0x2b,0x09,0x00,0xae,0xff,0x07,
0xb2,0xff,0x00,0xb4,0xff,0x00,0xb5,0xff,0x00,0xc3,0x01,0x00,0xc7,0xff,0xbf,
0xe7,0x08,0x00,0xf0,0x02,0x00
};
lib/minhook/src/hde/table64.h
Executable
+74
View File
@@ -0,0 +1,74 @@
/*
* Hacker Disassembler Engine 64 C
* Copyright (c) 2008-2009, Vyacheslav Patkov.
* All rights reserved.
*
*/
#define C_NONE 0x00
#define C_MODRM 0x01
#define C_IMM8 0x02
#define C_IMM16 0x04
#define C_IMM_P66 0x10
#define C_REL8 0x20
#define C_REL32 0x40
#define C_GROUP 0x80
#define C_ERROR 0xff
#define PRE_ANY 0x00
#define PRE_NONE 0x01
#define PRE_F2 0x02
#define PRE_F3 0x04
#define PRE_66 0x08
#define PRE_67 0x10
#define PRE_LOCK 0x20
#define PRE_SEG 0x40
#define PRE_ALL 0xff
#define DELTA_OPCODES 0x4a
#define DELTA_FPU_REG 0xfd
#define DELTA_FPU_MODRM 0x104
#define DELTA_PREFIXES 0x13c
#define DELTA_OP_LOCK_OK 0x1ae
#define DELTA_OP2_LOCK_OK 0x1c6
#define DELTA_OP_ONLY_MEM 0x1d8
#define DELTA_OP2_ONLY_MEM 0x1e7
unsigned char hde64_table[] = {
0xa5,0xaa,0xa5,0xb8,0xa5,0xaa,0xa5,0xaa,0xa5,0xb8,0xa5,0xb8,0xa5,0xb8,0xa5,
0xb8,0xc0,0xc0,0xc0,0xc0,0xc0,0xc0,0xc0,0xc0,0xac,0xc0,0xcc,0xc0,0xa1,0xa1,
0xa1,0xa1,0xb1,0xa5,0xa5,0xa6,0xc0,0xc0,0xd7,0xda,0xe0,0xc0,0xe4,0xc0,0xea,
0xea,0xe0,0xe0,0x98,0xc8,0xee,0xf1,0xa5,0xd3,0xa5,0xa5,0xa1,0xea,0x9e,0xc0,
0xc0,0xc2,0xc0,0xe6,0x03,0x7f,0x11,0x7f,0x01,0x7f,0x01,0x3f,0x01,0x01,0xab,
0x8b,0x90,0x64,0x5b,0x5b,0x5b,0x5b,0x5b,0x92,0x5b,0x5b,0x76,0x90,0x92,0x92,
0x5b,0x5b,0x5b,0x5b,0x5b,0x5b,0x5b,0x5b,0x5b,0x5b,0x5b,0x5b,0x6a,0x73,0x90,
0x5b,0x52,0x52,0x52,0x52,0x5b,0x5b,0x5b,0x5b,0x77,0x7c,0x77,0x85,0x5b,0x5b,
0x70,0x5b,0x7a,0xaf,0x76,0x76,0x5b,0x5b,0x5b,0x5b,0x5b,0x5b,0x5b,0x5b,0x5b,
0x5b,0x5b,0x86,0x01,0x03,0x01,0x04,0x03,0xd5,0x03,0xd5,0x03,0xcc,0x01,0xbc,
0x03,0xf0,0x03,0x03,0x04,0x00,0x50,0x50,0x50,0x50,0xff,0x20,0x20,0x20,0x20,
0x01,0x01,0x01,0x01,0xc4,0x02,0x10,0xff,0xff,0xff,0x01,0x00,0x03,0x11,0xff,
0x03,0xc4,0xc6,0xc8,0x02,0x10,0x00,0xff,0xcc,0x01,0x01,0x01,0x00,0x00,0x00,
0x00,0x01,0x01,0x03,0x01,0xff,0xff,0xc0,0xc2,0x10,0x11,0x02,0x03,0x01,0x01,
0x01,0xff,0xff,0xff,0x00,0x00,0x00,0xff,0x00,0x00,0xff,0xff,0xff,0xff,0x10,
0x10,0x10,0x10,0x02,0x10,0x00,0x00,0xc6,0xc8,0x02,0x02,0x02,0x02,0x06,0x00,
0x04,0x00,0x02,0xff,0x00,0xc0,0xc2,0x01,0x01,0x03,0x03,0x03,0xca,0x40,0x00,
0x0a,0x00,0x04,0x00,0x00,0x00,0x00,0x7f,0x00,0x33,0x01,0x00,0x00,0x00,0x00,
0x00,0x00,0xff,0xbf,0xff,0xff,0x00,0x00,0x00,0x00,0x07,0x00,0x00,0xff,0x00,
0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,
0x00,0x00,0x00,0xbf,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x7f,0x00,0x00,
0xff,0x40,0x40,0x40,0x40,0x41,0x49,0x40,0x40,0x40,0x40,0x4c,0x42,0x40,0x40,
0x40,0x40,0x40,0x40,0x40,0x40,0x4f,0x44,0x53,0x40,0x40,0x40,0x44,0x57,0x43,
0x5c,0x40,0x60,0x40,0x40,0x40,0x40,0x40,0x40,0x40,0x40,0x40,0x40,0x40,0x40,
0x40,0x40,0x64,0x66,0x6e,0x6b,0x40,0x40,0x6a,0x46,0x40,0x40,0x44,0x46,0x40,
0x40,0x5b,0x44,0x40,0x40,0x00,0x00,0x00,0x00,0x06,0x06,0x06,0x06,0x01,0x06,
0x06,0x02,0x06,0x06,0x00,0x06,0x00,0x0a,0x0a,0x00,0x00,0x00,0x02,0x07,0x07,
0x06,0x02,0x0d,0x06,0x06,0x06,0x0e,0x05,0x05,0x02,0x02,0x00,0x00,0x04,0x04,
0x04,0x04,0x05,0x06,0x06,0x06,0x00,0x00,0x00,0x0e,0x00,0x00,0x08,0x00,0x10,
0x00,0x18,0x00,0x20,0x00,0x28,0x00,0x30,0x00,0x80,0x01,0x82,0x01,0x86,0x00,
0xf6,0xcf,0xfe,0x3f,0xab,0x00,0xb0,0x00,0xb1,0x00,0xb3,0x00,0xba,0xf8,0xbb,
0x00,0xc0,0x00,0xc1,0x00,0xc7,0xbf,0x62,0xff,0x00,0x8d,0xff,0x00,0xc4,0xff,
0x00,0xc5,0xff,0x00,0xff,0xff,0xeb,0x01,0xff,0x0e,0x12,0x08,0x00,0x13,0x09,
0x00,0x16,0x08,0x00,0x17,0x09,0x00,0x2b,0x09,0x00,0xae,0xff,0x07,0xb2,0xff,
0x00,0xb4,0xff,0x00,0xb5,0xff,0x00,0xc3,0x01,0x00,0xc7,0xff,0xbf,0xe7,0x08,
0x00,0xf0,0x02,0x00
};
lib/minhook/src/hook.c
Executable
+939
View File
@@ -0,0 +1,939 @@
/*
* MinHook - The Minimalistic API Hooking Library for x64/x86
* Copyright (C) 2009-2017 Tsuda Kageyu.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER
* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <windows.h>
#include <tlhelp32.h>
#include <limits.h>
#include "../include/MinHook.h"
#include "buffer.h"
#include "trampoline.h"
#ifndef ARRAYSIZE
#define ARRAYSIZE(A) (sizeof(A)/sizeof((A)[0]))
#endif
// Initial capacity of the HOOK_ENTRY buffer.
#define INITIAL_HOOK_CAPACITY 32
// Initial capacity of the thread IDs buffer.
#define INITIAL_THREAD_CAPACITY 128
// Special hook position values.
#define INVALID_HOOK_POS UINT_MAX
#define ALL_HOOKS_POS UINT_MAX
// Freeze() action argument defines.
#define ACTION_DISABLE 0
#define ACTION_ENABLE 1
#define ACTION_APPLY_QUEUED 2
// Thread access rights for suspending/resuming threads.
#define THREAD_ACCESS \
(THREAD_SUSPEND_RESUME | THREAD_GET_CONTEXT | THREAD_QUERY_INFORMATION | THREAD_SET_CONTEXT)
// Hook information.
typedef struct _HOOK_ENTRY
{
LPVOID pTarget; // Address of the target function.
LPVOID pDetour; // Address of the detour or relay function.
LPVOID pTrampoline; // Address of the trampoline function.
UINT8 backup[8]; // Original prologue of the target function.
UINT8 patchAbove : 1; // Uses the hot patch area.
UINT8 isEnabled : 1; // Enabled.
UINT8 queueEnable : 1; // Queued for enabling/disabling when != isEnabled.
UINT nIP : 4; // Count of the instruction boundaries.
UINT8 oldIPs[8]; // Instruction boundaries of the target function.
UINT8 newIPs[8]; // Instruction boundaries of the trampoline function.
} HOOK_ENTRY, *PHOOK_ENTRY;
// Suspended threads for Freeze()/Unfreeze().
typedef struct _FROZEN_THREADS
{
LPDWORD pItems; // Data heap
UINT capacity; // Size of allocated data heap, items
UINT size; // Actual number of data items
} FROZEN_THREADS, *PFROZEN_THREADS;
//-------------------------------------------------------------------------
// Global Variables:
//-------------------------------------------------------------------------
// Spin lock flag for EnterSpinLock()/LeaveSpinLock().
static volatile LONG g_isLocked = FALSE;
// Private heap handle. If not NULL, this library is initialized.
static HANDLE g_hHeap = NULL;
// Hook entries.
static struct
{
PHOOK_ENTRY pItems; // Data heap
UINT capacity; // Size of allocated data heap, items
UINT size; // Actual number of data items
} g_hooks;
//-------------------------------------------------------------------------
// Returns INVALID_HOOK_POS if not found.
static UINT FindHookEntry(LPVOID pTarget)
{
UINT i;
for (i = 0; i < g_hooks.size; ++i)
{
if ((ULONG_PTR)pTarget == (ULONG_PTR)g_hooks.pItems[i].pTarget)
return i;
}
return INVALID_HOOK_POS;
}
//-------------------------------------------------------------------------
static PHOOK_ENTRY AddHookEntry()
{
if (g_hooks.pItems == NULL)
{
g_hooks.capacity = INITIAL_HOOK_CAPACITY;
g_hooks.pItems = (PHOOK_ENTRY)HeapAlloc(
g_hHeap, 0, g_hooks.capacity * sizeof(HOOK_ENTRY));
if (g_hooks.pItems == NULL)
return NULL;
}
else if (g_hooks.size >= g_hooks.capacity)
{
PHOOK_ENTRY p = (PHOOK_ENTRY)HeapReAlloc(
g_hHeap, 0, g_hooks.pItems, (g_hooks.capacity * 2) * sizeof(HOOK_ENTRY));
if (p == NULL)
return NULL;
g_hooks.capacity *= 2;
g_hooks.pItems = p;
}
return &g_hooks.pItems[g_hooks.size++];
}
//-------------------------------------------------------------------------
static VOID DeleteHookEntry(UINT pos)
{
if (pos < g_hooks.size - 1)
g_hooks.pItems[pos] = g_hooks.pItems[g_hooks.size - 1];
g_hooks.size--;
if (g_hooks.capacity / 2 >= INITIAL_HOOK_CAPACITY && g_hooks.capacity / 2 >= g_hooks.size)
{
PHOOK_ENTRY p = (PHOOK_ENTRY)HeapReAlloc(
g_hHeap, 0, g_hooks.pItems, (g_hooks.capacity / 2) * sizeof(HOOK_ENTRY));
if (p == NULL)
return;
g_hooks.capacity /= 2;
g_hooks.pItems = p;
}
}
//-------------------------------------------------------------------------
static DWORD_PTR FindOldIP(PHOOK_ENTRY pHook, DWORD_PTR ip)
{
UINT i;
if (pHook->patchAbove && ip == ((DWORD_PTR)pHook->pTarget - sizeof(JMP_REL)))
return (DWORD_PTR)pHook->pTarget;
for (i = 0; i < pHook->nIP; ++i)
{
if (ip == ((DWORD_PTR)pHook->pTrampoline + pHook->newIPs[i]))
return (DWORD_PTR)pHook->pTarget + pHook->oldIPs[i];
}
#if defined(_M_X64) || defined(__x86_64__)
// Check relay function.
if (ip == (DWORD_PTR)pHook->pDetour)
return (DWORD_PTR)pHook->pTarget;
#endif
return 0;
}
//-------------------------------------------------------------------------
static DWORD_PTR FindNewIP(PHOOK_ENTRY pHook, DWORD_PTR ip)
{
UINT i;
for (i = 0; i < pHook->nIP; ++i)
{
if (ip == ((DWORD_PTR)pHook->pTarget + pHook->oldIPs[i]))
return (DWORD_PTR)pHook->pTrampoline + pHook->newIPs[i];
}
return 0;
}
//-------------------------------------------------------------------------
static VOID ProcessThreadIPs(HANDLE hThread, UINT pos, UINT action)
{
// If the thread suspended in the overwritten area,
// move IP to the proper address.
CONTEXT c;
#if defined(_M_X64) || defined(__x86_64__)
DWORD64 *pIP = &c.Rip;
#else
DWORD *pIP = &c.Eip;
#endif
UINT count;
c.ContextFlags = CONTEXT_CONTROL;
if (!GetThreadContext(hThread, &c))
return;
if (pos == ALL_HOOKS_POS)
{
pos = 0;
count = g_hooks.size;
}
else
{
count = pos + 1;
}
for (; pos < count; ++pos)
{
PHOOK_ENTRY pHook = &g_hooks.pItems[pos];
BOOL enable;
DWORD_PTR ip;
switch (action)
{
case ACTION_DISABLE:
enable = FALSE;
break;
case ACTION_ENABLE:
enable = TRUE;
break;
default: // ACTION_APPLY_QUEUED
enable = pHook->queueEnable;
break;
}
if (pHook->isEnabled == enable)
continue;
if (enable)
ip = FindNewIP(pHook, *pIP);
else
ip = FindOldIP(pHook, *pIP);
if (ip != 0)
{
*pIP = ip;
SetThreadContext(hThread, &c);
}
}
}
//-------------------------------------------------------------------------
static BOOL EnumerateThreads(PFROZEN_THREADS pThreads)
{
BOOL succeeded = FALSE;
HANDLE hSnapshot = CreateToolhelp32Snapshot(TH32CS_SNAPTHREAD, 0);
if (hSnapshot != INVALID_HANDLE_VALUE)
{
THREADENTRY32 te;
te.dwSize = sizeof(THREADENTRY32);
if (Thread32First(hSnapshot, &te))
{
succeeded = TRUE;
do
{
if (te.dwSize >= (FIELD_OFFSET(THREADENTRY32, th32OwnerProcessID) + sizeof(DWORD))
&& te.th32OwnerProcessID == GetCurrentProcessId()
&& te.th32ThreadID != GetCurrentThreadId())
{
if (pThreads->pItems == NULL)
{
pThreads->capacity = INITIAL_THREAD_CAPACITY;
pThreads->pItems
= (LPDWORD)HeapAlloc(g_hHeap, 0, pThreads->capacity * sizeof(DWORD));
if (pThreads->pItems == NULL)
{
succeeded = FALSE;
break;
}
}
else if (pThreads->size >= pThreads->capacity)
{
LPDWORD p;
pThreads->capacity *= 2;
p = (LPDWORD)HeapReAlloc(
g_hHeap, 0, pThreads->pItems, pThreads->capacity * sizeof(DWORD));
if (p == NULL)
{
succeeded = FALSE;
break;
}
pThreads->pItems = p;
}
pThreads->pItems[pThreads->size++] = te.th32ThreadID;
}
te.dwSize = sizeof(THREADENTRY32);
} while (Thread32Next(hSnapshot, &te));
if (succeeded && GetLastError() != ERROR_NO_MORE_FILES)
succeeded = FALSE;
if (!succeeded && pThreads->pItems != NULL)
{
HeapFree(g_hHeap, 0, pThreads->pItems);
pThreads->pItems = NULL;
}
}
CloseHandle(hSnapshot);
}
return succeeded;
}
//-------------------------------------------------------------------------
static MH_STATUS Freeze(PFROZEN_THREADS pThreads, UINT pos, UINT action)
{
MH_STATUS status = MH_OK;
pThreads->pItems = NULL;
pThreads->capacity = 0;
pThreads->size = 0;
if (!EnumerateThreads(pThreads))
{
status = MH_ERROR_MEMORY_ALLOC;
}
else if (pThreads->pItems != NULL)
{
UINT i;
for (i = 0; i < pThreads->size; ++i)
{
HANDLE hThread = OpenThread(THREAD_ACCESS, FALSE, pThreads->pItems[i]);
BOOL suspended = FALSE;
if (hThread != NULL)
{
DWORD result = SuspendThread(hThread);
if (result != 0xFFFFFFFF)
{
suspended = TRUE;
ProcessThreadIPs(hThread, pos, action);
}
CloseHandle(hThread);
}
if (!suspended)
{
// Mark thread as not suspended, so it's not resumed later on.
pThreads->pItems[i] = 0;
}
}
}
return status;
}
//-------------------------------------------------------------------------
static VOID Unfreeze(PFROZEN_THREADS pThreads)
{
if (pThreads->pItems != NULL)
{
UINT i;
for (i = 0; i < pThreads->size; ++i)
{
DWORD threadId = pThreads->pItems[i];
if (threadId != 0)
{
HANDLE hThread = OpenThread(THREAD_ACCESS, FALSE, threadId);
if (hThread != NULL)
{
ResumeThread(hThread);
CloseHandle(hThread);
}
}
}
HeapFree(g_hHeap, 0, pThreads->pItems);
}
}
//-------------------------------------------------------------------------
static MH_STATUS EnableHookLL(UINT pos, BOOL enable)
{
PHOOK_ENTRY pHook = &g_hooks.pItems[pos];
DWORD oldProtect;
SIZE_T patchSize = sizeof(JMP_REL);
LPBYTE pPatchTarget = (LPBYTE)pHook->pTarget;
if (pHook->patchAbove)
{
pPatchTarget -= sizeof(JMP_REL);
patchSize += sizeof(JMP_REL_SHORT);
}
if (!VirtualProtect(pPatchTarget, patchSize, PAGE_EXECUTE_READWRITE, &oldProtect))
return MH_ERROR_MEMORY_PROTECT;
if (enable)
{
PJMP_REL pJmp = (PJMP_REL)pPatchTarget;
pJmp->opcode = 0xE9;
pJmp->operand = (INT32)((LPBYTE)pHook->pDetour - (pPatchTarget + sizeof(JMP_REL)));
if (pHook->patchAbove)
{
PJMP_REL_SHORT pShortJmp = (PJMP_REL_SHORT)pHook->pTarget;
pShortJmp->opcode = 0xEB;
pShortJmp->operand = (INT8)(0 - (sizeof(JMP_REL_SHORT) + sizeof(JMP_REL)));
}
}
else
{
if (pHook->patchAbove)
memcpy(pPatchTarget, pHook->backup, sizeof(JMP_REL) + sizeof(JMP_REL_SHORT));
else
memcpy(pPatchTarget, pHook->backup, sizeof(JMP_REL));
}
VirtualProtect(pPatchTarget, patchSize, oldProtect, &oldProtect);
// Just-in-case measure.
FlushInstructionCache(GetCurrentProcess(), pPatchTarget, patchSize);
pHook->isEnabled = enable;
pHook->queueEnable = enable;
return MH_OK;
}
//-------------------------------------------------------------------------
static MH_STATUS EnableAllHooksLL(BOOL enable)
{
MH_STATUS status = MH_OK;
UINT i, first = INVALID_HOOK_POS;
for (i = 0; i < g_hooks.size; ++i)
{
if (g_hooks.pItems[i].isEnabled != enable)
{
first = i;
break;
}
}
if (first != INVALID_HOOK_POS)
{
FROZEN_THREADS threads;
status = Freeze(&threads, ALL_HOOKS_POS, enable ? ACTION_ENABLE : ACTION_DISABLE);
if (status == MH_OK)
{
for (i = first; i < g_hooks.size; ++i)
{
if (g_hooks.pItems[i].isEnabled != enable)
{
status = EnableHookLL(i, enable);
if (status != MH_OK)
break;
}
}
Unfreeze(&threads);
}
}
return status;
}
//-------------------------------------------------------------------------
static VOID EnterSpinLock(VOID)
{
SIZE_T spinCount = 0;
// Wait until the flag is FALSE.
while (InterlockedCompareExchange(&g_isLocked, TRUE, FALSE) != FALSE)
{
// No need to generate a memory barrier here, since InterlockedCompareExchange()
// generates a full memory barrier itself.
// Prevent the loop from being too busy.
if (spinCount < 32)
Sleep(0);
else
Sleep(1);
spinCount++;
}
}
//-------------------------------------------------------------------------
static VOID LeaveSpinLock(VOID)
{
// No need to generate a memory barrier here, since InterlockedExchange()
// generates a full memory barrier itself.
InterlockedExchange(&g_isLocked, FALSE);
}
//-------------------------------------------------------------------------
MH_STATUS WINAPI MH_Initialize(VOID)
{
MH_STATUS status = MH_OK;
EnterSpinLock();
if (g_hHeap == NULL)
{
g_hHeap = HeapCreate(0, 0, 0);
if (g_hHeap != NULL)
{
// Initialize the internal function buffer.
InitializeBuffer();
}
else
{
status = MH_ERROR_MEMORY_ALLOC;
}
}
else
{
status = MH_ERROR_ALREADY_INITIALIZED;
}
LeaveSpinLock();
return status;
}
//-------------------------------------------------------------------------
MH_STATUS WINAPI MH_Uninitialize(VOID)
{
MH_STATUS status = MH_OK;
EnterSpinLock();
if (g_hHeap != NULL)
{
status = EnableAllHooksLL(FALSE);
if (status == MH_OK)
{
// Free the internal function buffer.
// HeapFree is actually not required, but some tools detect a false
// memory leak without HeapFree.
UninitializeBuffer();
HeapFree(g_hHeap, 0, g_hooks.pItems);
HeapDestroy(g_hHeap);
g_hHeap = NULL;
g_hooks.pItems = NULL;
g_hooks.capacity = 0;
g_hooks.size = 0;
}
}
else
{
status = MH_ERROR_NOT_INITIALIZED;
}
LeaveSpinLock();
return status;
}
//-------------------------------------------------------------------------
MH_STATUS WINAPI MH_CreateHook(LPVOID pTarget, LPVOID pDetour, LPVOID *ppOriginal)
{
MH_STATUS status = MH_OK;
EnterSpinLock();
if (g_hHeap != NULL)
{
if (IsExecutableAddress(pTarget) && IsExecutableAddress(pDetour))
{
UINT pos = FindHookEntry(pTarget);
if (pos == INVALID_HOOK_POS)
{
LPVOID pBuffer = AllocateBuffer(pTarget);
if (pBuffer != NULL)
{
TRAMPOLINE ct;
ct.pTarget = pTarget;
ct.pDetour = pDetour;
ct.pTrampoline = pBuffer;
if (CreateTrampolineFunction(&ct))
{
PHOOK_ENTRY pHook = AddHookEntry();
if (pHook != NULL)
{
pHook->pTarget = ct.pTarget;
#if defined(_M_X64) || defined(__x86_64__)
pHook->pDetour = ct.pRelay;
#else
pHook->pDetour = ct.pDetour;
#endif
pHook->pTrampoline = ct.pTrampoline;
pHook->patchAbove = ct.patchAbove;
pHook->isEnabled = FALSE;
pHook->queueEnable = FALSE;
pHook->nIP = ct.nIP;
memcpy(pHook->oldIPs, ct.oldIPs, ARRAYSIZE(ct.oldIPs));
memcpy(pHook->newIPs, ct.newIPs, ARRAYSIZE(ct.newIPs));
// Back up the target function.
if (ct.patchAbove)
{
memcpy(
pHook->backup,
(LPBYTE)pTarget - sizeof(JMP_REL),
sizeof(JMP_REL) + sizeof(JMP_REL_SHORT));
}
else
{
memcpy(pHook->backup, pTarget, sizeof(JMP_REL));
}
if (ppOriginal != NULL)
*ppOriginal = pHook->pTrampoline;
}
else
{
status = MH_ERROR_MEMORY_ALLOC;
}
}
else
{
status = MH_ERROR_UNSUPPORTED_FUNCTION;
}
if (status != MH_OK)
{
FreeBuffer(pBuffer);
}
}
else
{
status = MH_ERROR_MEMORY_ALLOC;
}
}
else
{
status = MH_ERROR_ALREADY_CREATED;
}
}
else
{
status = MH_ERROR_NOT_EXECUTABLE;
}
}
else
{
status = MH_ERROR_NOT_INITIALIZED;
}
LeaveSpinLock();
return status;
}
//-------------------------------------------------------------------------
MH_STATUS WINAPI MH_RemoveHook(LPVOID pTarget)
{
MH_STATUS status = MH_OK;
EnterSpinLock();
if (g_hHeap != NULL)
{
UINT pos = FindHookEntry(pTarget);
if (pos != INVALID_HOOK_POS)
{
if (g_hooks.pItems[pos].isEnabled)
{
FROZEN_THREADS threads;
status = Freeze(&threads, pos, ACTION_DISABLE);
if (status == MH_OK)
{
status = EnableHookLL(pos, FALSE);
Unfreeze(&threads);
}
}
if (status == MH_OK)
{
FreeBuffer(g_hooks.pItems[pos].pTrampoline);
DeleteHookEntry(pos);
}
}
else
{
status = MH_ERROR_NOT_CREATED;
}
}
else
{
status = MH_ERROR_NOT_INITIALIZED;
}
LeaveSpinLock();
return status;
}
//-------------------------------------------------------------------------
static MH_STATUS EnableHook(LPVOID pTarget, BOOL enable)
{
MH_STATUS status = MH_OK;
EnterSpinLock();
if (g_hHeap != NULL)
{
if (pTarget == MH_ALL_HOOKS)
{
status = EnableAllHooksLL(enable);
}
else
{
UINT pos = FindHookEntry(pTarget);
if (pos != INVALID_HOOK_POS)
{
if (g_hooks.pItems[pos].isEnabled != enable)
{
FROZEN_THREADS threads;
status = Freeze(&threads, pos, ACTION_ENABLE);
if (status == MH_OK)
{
status = EnableHookLL(pos, enable);
Unfreeze(&threads);
}
}
else
{
status = enable ? MH_ERROR_ENABLED : MH_ERROR_DISABLED;
}
}
else
{
status = MH_ERROR_NOT_CREATED;
}
}
}
else
{
status = MH_ERROR_NOT_INITIALIZED;
}
LeaveSpinLock();
return status;
}
//-------------------------------------------------------------------------
MH_STATUS WINAPI MH_EnableHook(LPVOID pTarget)
{
return EnableHook(pTarget, TRUE);
}
//-------------------------------------------------------------------------
MH_STATUS WINAPI MH_DisableHook(LPVOID pTarget)
{
return EnableHook(pTarget, FALSE);
}
//-------------------------------------------------------------------------
static MH_STATUS QueueHook(LPVOID pTarget, BOOL queueEnable)
{
MH_STATUS status = MH_OK;
EnterSpinLock();
if (g_hHeap != NULL)
{
if (pTarget == MH_ALL_HOOKS)
{
UINT i;
for (i = 0; i < g_hooks.size; ++i)
g_hooks.pItems[i].queueEnable = queueEnable;
}
else
{
UINT pos = FindHookEntry(pTarget);
if (pos != INVALID_HOOK_POS)
{
g_hooks.pItems[pos].queueEnable = queueEnable;
}
else
{
status = MH_ERROR_NOT_CREATED;
}
}
}
else
{
status = MH_ERROR_NOT_INITIALIZED;
}
LeaveSpinLock();
return status;
}
//-------------------------------------------------------------------------
MH_STATUS WINAPI MH_QueueEnableHook(LPVOID pTarget)
{
return QueueHook(pTarget, TRUE);
}
//-------------------------------------------------------------------------
MH_STATUS WINAPI MH_QueueDisableHook(LPVOID pTarget)
{
return QueueHook(pTarget, FALSE);
}
//-------------------------------------------------------------------------
MH_STATUS WINAPI MH_ApplyQueued(VOID)
{
MH_STATUS status = MH_OK;
UINT i, first = INVALID_HOOK_POS;
EnterSpinLock();
if (g_hHeap != NULL)
{
for (i = 0; i < g_hooks.size; ++i)
{
if (g_hooks.pItems[i].isEnabled != g_hooks.pItems[i].queueEnable)
{
first = i;
break;
}
}
if (first != INVALID_HOOK_POS)
{
FROZEN_THREADS threads;
status = Freeze(&threads, ALL_HOOKS_POS, ACTION_APPLY_QUEUED);
if (status == MH_OK)
{
for (i = first; i < g_hooks.size; ++i)
{
PHOOK_ENTRY pHook = &g_hooks.pItems[i];
if (pHook->isEnabled != pHook->queueEnable)
{
status = EnableHookLL(i, pHook->queueEnable);
if (status != MH_OK)
break;
}
}
Unfreeze(&threads);
}
}
}
else
{
status = MH_ERROR_NOT_INITIALIZED;
}
LeaveSpinLock();
return status;
}
//-------------------------------------------------------------------------
MH_STATUS WINAPI MH_CreateHookApiEx(
LPCWSTR pszModule, LPCSTR pszProcName, LPVOID pDetour,
LPVOID *ppOriginal, LPVOID *ppTarget)
{
HMODULE hModule;
LPVOID pTarget;
hModule = GetModuleHandleW(pszModule);
if (hModule == NULL)
return MH_ERROR_MODULE_NOT_FOUND;
pTarget = (LPVOID)GetProcAddress(hModule, pszProcName);
if (pTarget == NULL)
return MH_ERROR_FUNCTION_NOT_FOUND;
if (ppTarget != NULL)
*ppTarget = pTarget;
return MH_CreateHook(pTarget, pDetour, ppOriginal);
}
//-------------------------------------------------------------------------
MH_STATUS WINAPI MH_CreateHookApi(
LPCWSTR pszModule, LPCSTR pszProcName, LPVOID pDetour, LPVOID *ppOriginal)
{
return MH_CreateHookApiEx(pszModule, pszProcName, pDetour, ppOriginal, NULL);
}
//-------------------------------------------------------------------------
const char *WINAPI MH_StatusToString(MH_STATUS status)
{
#define MH_ST2STR(x) \
case x: \
return #x;
switch (status) {
MH_ST2STR(MH_UNKNOWN)
MH_ST2STR(MH_OK)
MH_ST2STR(MH_ERROR_ALREADY_INITIALIZED)
MH_ST2STR(MH_ERROR_NOT_INITIALIZED)
MH_ST2STR(MH_ERROR_ALREADY_CREATED)
MH_ST2STR(MH_ERROR_NOT_CREATED)
MH_ST2STR(MH_ERROR_ENABLED)
MH_ST2STR(MH_ERROR_DISABLED)
MH_ST2STR(MH_ERROR_NOT_EXECUTABLE)
MH_ST2STR(MH_ERROR_UNSUPPORTED_FUNCTION)
MH_ST2STR(MH_ERROR_MEMORY_ALLOC)
MH_ST2STR(MH_ERROR_MEMORY_PROTECT)
MH_ST2STR(MH_ERROR_MODULE_NOT_FOUND)
MH_ST2STR(MH_ERROR_FUNCTION_NOT_FOUND)
}
#undef MH_ST2STR
return "(unknown)";
}
lib/minhook/src/trampoline.c
Executable
+320
View File
@@ -0,0 +1,320 @@
/*
* MinHook - The Minimalistic API Hooking Library for x64/x86
* Copyright (C) 2009-2017 Tsuda Kageyu.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER
* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <windows.h>
#if defined(_MSC_VER) && !defined(MINHOOK_DISABLE_INTRINSICS)
#define ALLOW_INTRINSICS
#include <intrin.h>
#endif
#ifndef ARRAYSIZE
#define ARRAYSIZE(A) (sizeof(A)/sizeof((A)[0]))
#endif
#if defined(_M_X64) || defined(__x86_64__)
#include "./hde/hde64.h"
typedef hde64s HDE;
#define HDE_DISASM(code, hs) hde64_disasm(code, hs)
#else
#include "./hde/hde32.h"
typedef hde32s HDE;
#define HDE_DISASM(code, hs) hde32_disasm(code, hs)
#endif
#include "trampoline.h"
#include "buffer.h"
// Maximum size of a trampoline function.
#if defined(_M_X64) || defined(__x86_64__)
#define TRAMPOLINE_MAX_SIZE (MEMORY_SLOT_SIZE - sizeof(JMP_ABS))
#else
#define TRAMPOLINE_MAX_SIZE MEMORY_SLOT_SIZE
#endif
//-------------------------------------------------------------------------
static BOOL IsCodePadding(LPBYTE pInst, UINT size)
{
UINT i;
if (pInst[0] != 0x00 && pInst[0] != 0x90 && pInst[0] != 0xCC)
return FALSE;
for (i = 1; i < size; ++i)
{
if (pInst[i] != pInst[0])
return FALSE;
}
return TRUE;
}
//-------------------------------------------------------------------------
BOOL CreateTrampolineFunction(PTRAMPOLINE ct)
{
#if defined(_M_X64) || defined(__x86_64__)
CALL_ABS call = {
0xFF, 0x15, 0x00000002, // FF15 00000002: CALL [RIP+8]
0xEB, 0x08, // EB 08: JMP +10
0x0000000000000000ULL // Absolute destination address
};
JMP_ABS jmp = {
0xFF, 0x25, 0x00000000, // FF25 00000000: JMP [RIP+6]
0x0000000000000000ULL // Absolute destination address
};
JCC_ABS jcc = {
0x70, 0x0E, // 7* 0E: J** +16
0xFF, 0x25, 0x00000000, // FF25 00000000: JMP [RIP+6]
0x0000000000000000ULL // Absolute destination address
};
#else
CALL_REL call = {
0xE8, // E8 xxxxxxxx: CALL +5+xxxxxxxx
0x00000000 // Relative destination address
};
JMP_REL jmp = {
0xE9, // E9 xxxxxxxx: JMP +5+xxxxxxxx
0x00000000 // Relative destination address
};
JCC_REL jcc = {
0x0F, 0x80, // 0F8* xxxxxxxx: J** +6+xxxxxxxx
0x00000000 // Relative destination address
};
#endif
UINT8 oldPos = 0;
UINT8 newPos = 0;
ULONG_PTR jmpDest = 0; // Destination address of an internal jump.
BOOL finished = FALSE; // Is the function completed?
#if defined(_M_X64) || defined(__x86_64__)
UINT8 instBuf[16];
#endif
ct->patchAbove = FALSE;
ct->nIP = 0;
do
{
HDE hs;
UINT copySize;
LPVOID pCopySrc;
ULONG_PTR pOldInst = (ULONG_PTR)ct->pTarget + oldPos;
ULONG_PTR pNewInst = (ULONG_PTR)ct->pTrampoline + newPos;
copySize = HDE_DISASM((LPVOID)pOldInst, &hs);
if (hs.flags & F_ERROR)
return FALSE;
pCopySrc = (LPVOID)pOldInst;
if (oldPos >= sizeof(JMP_REL))
{
// The trampoline function is long enough.
// Complete the function with the jump to the target function.
#if defined(_M_X64) || defined(__x86_64__)
jmp.address = pOldInst;
#else
jmp.operand = (INT32)(pOldInst - (pNewInst + sizeof(jmp)));
#endif
pCopySrc = &jmp;
copySize = sizeof(jmp);
finished = TRUE;
}
#if defined(_M_X64) || defined(__x86_64__)
else if ((hs.modrm & 0xC7) == 0x05)
{
// Instructions using RIP relative addressing. (ModR/M = 00???101B)
// Modify the RIP relative address.
PUINT32 pRelAddr;
// Avoid using memcpy to reduce the footprint.
#ifndef ALLOW_INTRINSICS
memcpy(instBuf, (LPBYTE)pOldInst, copySize);
#else
__movsb(instBuf, (LPBYTE)pOldInst, copySize);
#endif
pCopySrc = instBuf;
// Relative address is stored at (instruction length - immediate value length - 4).
pRelAddr = (PUINT32)(instBuf + hs.len - ((hs.flags & 0x3C) >> 2) - 4);
*pRelAddr
= (UINT32)((pOldInst + hs.len + (INT32)hs.disp.disp32) - (pNewInst + hs.len));
// Complete the function if JMP (FF /4).
if (hs.opcode == 0xFF && hs.modrm_reg == 4)
finished = TRUE;
}
#endif
else if (hs.opcode == 0xE8)
{
// Direct relative CALL
ULONG_PTR dest = pOldInst + hs.len + (INT32)hs.imm.imm32;
#if defined(_M_X64) || defined(__x86_64__)
call.address = dest;
#else
call.operand = (INT32)(dest - (pNewInst + sizeof(call)));
#endif
pCopySrc = &call;
copySize = sizeof(call);
}
else if ((hs.opcode & 0xFD) == 0xE9)
{
// Direct relative JMP (EB or E9)
ULONG_PTR dest = pOldInst + hs.len;
if (hs.opcode == 0xEB) // isShort jmp
dest += (INT8)hs.imm.imm8;
else
dest += (INT32)hs.imm.imm32;
// Simply copy an internal jump.
if ((ULONG_PTR)ct->pTarget <= dest
&& dest < ((ULONG_PTR)ct->pTarget + sizeof(JMP_REL)))
{
if (jmpDest < dest)
jmpDest = dest;
}
else
{
#if defined(_M_X64) || defined(__x86_64__)
jmp.address = dest;
#else
jmp.operand = (INT32)(dest - (pNewInst + sizeof(jmp)));
#endif
pCopySrc = &jmp;
copySize = sizeof(jmp);
// Exit the function if it is not in the branch.
finished = (pOldInst >= jmpDest);
}
}
else if ((hs.opcode & 0xF0) == 0x70
|| (hs.opcode & 0xFC) == 0xE0
|| (hs.opcode2 & 0xF0) == 0x80)
{
// Direct relative Jcc
ULONG_PTR dest = pOldInst + hs.len;
if ((hs.opcode & 0xF0) == 0x70 // Jcc
|| (hs.opcode & 0xFC) == 0xE0) // LOOPNZ/LOOPZ/LOOP/JECXZ
dest += (INT8)hs.imm.imm8;
else
dest += (INT32)hs.imm.imm32;
// Simply copy an internal jump.
if ((ULONG_PTR)ct->pTarget <= dest
&& dest < ((ULONG_PTR)ct->pTarget + sizeof(JMP_REL)))
{
if (jmpDest < dest)
jmpDest = dest;
}
else if ((hs.opcode & 0xFC) == 0xE0)
{
// LOOPNZ/LOOPZ/LOOP/JCXZ/JECXZ to the outside are not supported.
return FALSE;
}
else
{
UINT8 cond = ((hs.opcode != 0x0F ? hs.opcode : hs.opcode2) & 0x0F);
#if defined(_M_X64) || defined(__x86_64__)
// Invert the condition in x64 mode to simplify the conditional jump logic.
jcc.opcode = 0x71 ^ cond;
jcc.address = dest;
#else
jcc.opcode1 = 0x80 | cond;
jcc.operand = (INT32)(dest - (pNewInst + sizeof(jcc)));
#endif
pCopySrc = &jcc;
copySize = sizeof(jcc);
}
}
else if ((hs.opcode & 0xFE) == 0xC2)
{
// RET (C2 or C3)
// Complete the function if not in a branch.
finished = (pOldInst >= jmpDest);
}
// Can't alter the instruction length in a branch.
if (pOldInst < jmpDest && copySize != hs.len)
return FALSE;
// Trampoline function is too large.
if ((newPos + copySize) > TRAMPOLINE_MAX_SIZE)
return FALSE;
// Trampoline function has too many instructions.
if (ct->nIP >= ARRAYSIZE(ct->oldIPs))
return FALSE;
ct->oldIPs[ct->nIP] = oldPos;
ct->newIPs[ct->nIP] = newPos;
ct->nIP++;
// Avoid using memcpy to reduce the footprint.
#ifndef ALLOW_INTRINSICS
memcpy((LPBYTE)ct->pTrampoline + newPos, pCopySrc, copySize);
#else
__movsb((LPBYTE)ct->pTrampoline + newPos, (LPBYTE)pCopySrc, copySize);
#endif
newPos += copySize;
oldPos += hs.len;
} while (!finished);
// Is there enough place for a long jump?
if (oldPos < sizeof(JMP_REL)
&& !IsCodePadding((LPBYTE)ct->pTarget + oldPos, sizeof(JMP_REL) - oldPos))
{
// Is there enough place for a short jump?
if (oldPos < sizeof(JMP_REL_SHORT)
&& !IsCodePadding((LPBYTE)ct->pTarget + oldPos, sizeof(JMP_REL_SHORT) - oldPos))
{
return FALSE;
}
// Can we place the long jump above the function?
if (!IsExecutableAddress((LPBYTE)ct->pTarget - sizeof(JMP_REL)))
return FALSE;
if (!IsCodePadding((LPBYTE)ct->pTarget - sizeof(JMP_REL), sizeof(JMP_REL)))
return FALSE;
ct->patchAbove = TRUE;
}
#if defined(_M_X64) || defined(__x86_64__)
// Create a relay function.
jmp.address = (ULONG_PTR)ct->pDetour;
ct->pRelay = (LPBYTE)ct->pTrampoline + newPos;
memcpy(ct->pRelay, &jmp, sizeof(jmp));
#endif
return TRUE;
}
lib/minhook/src/trampoline.h
Executable
+105
View File
@@ -0,0 +1,105 @@
/*
* MinHook - The Minimalistic API Hooking Library for x64/x86
* Copyright (C) 2009-2017 Tsuda Kageyu.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER
* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#pragma once
#pragma pack(push, 1)
// Structs for writing x86/x64 instructions.
// 8-bit relative jump.
typedef struct _JMP_REL_SHORT
{
UINT8 opcode; // EB xx: JMP +2+xx
INT8 operand; // Relative destination address
} JMP_REL_SHORT, *PJMP_REL_SHORT;
// 32-bit direct relative jump/call.
typedef struct _JMP_REL
{
UINT8 opcode; // E9/E8 xxxxxxxx: JMP/CALL +5+xxxxxxxx
INT32 operand; // Relative destination address
} JMP_REL, *PJMP_REL, CALL_REL;
// 64-bit indirect absolute jump.
typedef struct _JMP_ABS
{
UINT8 opcode0; // FF25 00000000: JMP [+6]
UINT8 opcode1;
UINT32 dummy;
UINT64 address; // Absolute destination address
} JMP_ABS, *PJMP_ABS;
// 64-bit indirect absolute call.
typedef struct _CALL_ABS
{
UINT8 opcode0; // FF15 00000002: CALL [+6]
UINT8 opcode1;
UINT32 dummy0;
UINT8 dummy1; // EB 08: JMP +10
UINT8 dummy2;
UINT64 address; // Absolute destination address
} CALL_ABS;
// 32-bit direct relative conditional jumps.
typedef struct _JCC_REL
{
UINT8 opcode0; // 0F8* xxxxxxxx: J** +6+xxxxxxxx
UINT8 opcode1;
INT32 operand; // Relative destination address
} JCC_REL;
// 64bit indirect absolute conditional jumps that x64 lacks.
typedef struct _JCC_ABS
{
UINT8 opcode; // 7* 0E: J** +16
UINT8 dummy0;
UINT8 dummy1; // FF25 00000000: JMP [+6]
UINT8 dummy2;
UINT32 dummy3;
UINT64 address; // Absolute destination address
} JCC_ABS;
#pragma pack(pop)
typedef struct _TRAMPOLINE
{
LPVOID pTarget; // [In] Address of the target function.
LPVOID pDetour; // [In] Address of the detour function.
LPVOID pTrampoline; // [In] Buffer address for the trampoline and relay function.
#if defined(_M_X64) || defined(__x86_64__)
LPVOID pRelay; // [Out] Address of the relay function.
#endif
BOOL patchAbove; // [Out] Should use the hot patch area?
UINT nIP; // [Out] Number of the instruction boundaries.
UINT8 oldIPs[8]; // [Out] Instruction boundaries of the target function.
UINT8 newIPs[8]; // [Out] Instruction boundaries of the trampoline function.
} TRAMPOLINE, *PTRAMPOLINE;
BOOL CreateTrampolineFunction(PTRAMPOLINE ct);