Commit 646298f2 authored by Samer Afach's avatar Samer Afach

First commit

parents
WAV to MP3 converter
====================
This software converts WAV files to MP3 using libmp3lame in parallel. It's a command prompt simple tool that can be executed using:
`WAVConverter.exe <path>`
where `<path>` is the directory where WAV files are located. The tool supports multithreading using pthread, and uses a Thread Pool to distribute jobs among available threads. The program detects automatically the number of threads available in the computer. If it fails in that, it uses only 1 thread.
Compiling the program
---------------------
To build the program, you need qmake.
The program is written to be cross-platform and is tested on Windows 10 and Linux (Debian). On Windows, it's tested on Visual Studio 2015 and MinGW-w64 6.2.0.
The code requires libmp3lame and pthread to compile.
Included stuff
--------------
If you have the zipped package of the program, then it contains pthread for Visual Studio 2015, with the `time.h` fix, and the `/MT` flag used for multi-threaded static run-time library compilation. The same applies for libmp3lame. However, there are 3 copies of `liblame` provided. This is because pthread isn't available on Visual Studio, but available by default in all posix systems (including linux and the MinGW compiler). Every `lame` directory contains a static, precompiled `.a` files for linking.
There's a directory (build-win32) which contains a pre-built executables of the program for testing, which are fully statically linked and are runnable as is.
The program should be compiled with C++11 support.
Compiling on Windows
--------------------
If you're going to compile on Windows, just make sure that you're using compatible linking with the libraries provided. Using `/MD` linking with the available libraries won't work.
Compiling on Linux
------------------
The provided qmake file compiles is set to use a precompiled static version of the lame library (that can be found in the libs directory, which is compiled for Debian 8). You should change that if you want to include the version that comes with your linux distribution.
API
---
The programming API is made to be self-explanatory. Inside every header file, there's information on how to use the classes provided and how to use them and the expected behavior.
Compatible file types
---------------------
The program is made to be compatible with standard PCM WAV files. The files are expected to have a header that explains what they are. Supported files are files with integer data of depth 8/12/16/24/32 bits, and floating-point data of depth 32/64 bits. Other kinds of data is rejected. It's quite easy to modify these conditions in the source code, as everything is packaged to detect the file specs from a `struct` that contains all the information about the relevant file.
Samer Afach, Oct, 10 2016
\ No newline at end of file
QT -= core
QT -= gui
CONFIG += c++11
win32 {
win32-msvc* {
TARGET = WAVConverter-msvc
} else:win32-g++ {
TARGET = WAVConverter-mingw
}
else {
TARGET = WAVConverter
}
}
else {
TARGET = WAVConverter
}
CONFIG += console
CONFIG -= app_bundle
TEMPLATE = app
VPATH += src
VPATH += include
INCLUDEPATH += include
SOURCES += main.cpp \
ThreadPool.cpp \
LockGuard.cpp \
FilesTools.cpp \
SoundConversion.cpp \
WAVHeader.cpp
HEADERS += \
ThreadPool.h \
LockGuard.h \
FilesTools.h \
SoundConversion.h \
WAVHeader.h
win32 {
DEFINES += "WIN32"
win32-msvc* {
#Visual Studio compiler options
INCLUDEPATH += "../libs/pthreads.2/"
#linking of pthread
LIBS += -L../libs/pthreads.2/
LIBS += -lpthreadVC2
LIBS += -lpthreadVC2d
DEFINES += "PTW32_STATIC_LIB"
#linking of libmp3lame
INCLUDEPATH += ../libs/lame-msvc/include
LIBS += -L../libs/lame-msvc/output/
LIBS += -llibmp3lame-static
} else {
#MinGW options
INCLUDEPATH += ../libs/lame-mingw/include
win32:LIBS += -L../libs/lame-mingw/libmp3lame/.libs/
LIBS += -lmp3lame
}
}
#liblame includes for linux and pthread linking
unix {
LIBS += -L../libs/lame-linux/libmp3lame/.libs/
LIBS += -lmp3lame
INCLUDEPATH += ../libs/lame-linux/include
QMAKE_CXXFLAGS += -pthread
}
#ifndef PATHTOOLS_H
#define PATHTOOLS_H
#include <string>
#include <vector>
#include <fstream>
#include <regex>
#ifdef _MSC_VER
#include <Windows.h>
#include <atlbase.h>
#else
#include <dirent.h>
#endif
/**
* @brief FileSize
* returns the size of the file given in the parameters
* @param filename
* @return
* returns the size of the file given in bytes. On failure to find the file, -1 is returned
*/
long long FileSize(const std::string& filename);
/**
* @brief GetDirSeparator
* returns the correct directory separator based on the operating system
* @return
* returns / for *nix systems and \ for windows.
*/
char GetDirSeparator();
/**
* @brief AddTailSlashToPath
* Adds a tailing slash at the end of a path. This helps to ensure that dirs are not treated as files
* @param path
*
*/
void AddTailSlashToPath(std::string& path);
/**
* @brief FixPathSlashes
* Converts slashes in a path to the correct; e.g, every / becomes \\ in Windows.
* @param path
*/
void FixPathSlashes(std::string& path);
/**
* @brief JoinPath
* Joins multiple paths together and separates them by the correct separator (slashes)
* based on the operating system
* @param params
* An initializer list with paths to be joined. E.g., {pathA, pathB, ...}
* @return
* Returns joined path
*/
std::string JoinPath(const std::initializer_list<std::string>& params);
/**
* @brief IsValidFile
* Checks if a file is valid
* @param path
* path to file
* @return
* returns true if the file is valid, false otherwise
*/
bool IsValidFile(const std::string& path);
/**
* @brief IsValidDir
* Checks if a directory is valid
* @param path
* @return
* returns true if the directory is valid, false otherwise
*/
bool IsValidDir(const std::string& path);
/**
* @brief GetFilesInDirectory
* Gives a list of files in the directory.
* It's possible to filter files with a regex entry.
* @param path_p
* Path to list files from
* @param regex_filter
* Regex to filter files found with
* @return
* returns a std::vector<std::string> of files found there with full path.
* Full path is given path + file names, not absolute path.
*/
std::vector<std::string> GetFilesInDirectory(const std::string& path_p, const std::string& regex_filter = std::string(".*"));
//Following is a way to detect endianness of a computer by setting a known value in binary
//and re-reading it after reinterpreting it
union endian_tester
{
std::uint32_t n;
std::uint8_t p[4];
};
const endian_tester endian_sample = {0x01020304}; // this initializes .n
enum hl_endianness : uint32_t {
HL_LITTLE_ENDIAN = 0x03020100,
HL_BIG_ENDIAN = 0x00010203,
HL_PDP_ENDIAN = 0x01000302,
HL_UNKNOWN_ENDIAN = 0xFFFFFFFF
};
hl_endianness GetThisComputerEndianOrder();
std::string GetThisComputerEndianOrderAsString();
template <typename T>
void SwapEndianness(T& var)
{
char* varSwapped = new char[sizeof(var)];
for(long i = 0; i < static_cast<long>(sizeof(var)); i++)
varSwapped[sizeof(var) - 1 - i] = ((char*)(&var))[i];
for(long i = 0; i < static_cast<long>(sizeof(var)); i++)
((char*)(&var))[i] = varSwapped[i];
delete[] varSwapped;
}
#define HL_ENDIANNESS getEndianOrder()
template <typename T>
T get_value_with_little_endianness(T val)
{
T processed_val = val;
if(GetThisComputerEndianOrder() == HL_BIG_ENDIAN)
{
SwapEndianness(processed_val);
}
else if(GetThisComputerEndianOrder() == HL_PDP_ENDIAN ||
GetThisComputerEndianOrder() == HL_UNKNOWN_ENDIAN)
{
throw std::runtime_error(std::string(
std::string("This computer has unsupported endianness: ") +
GetThisComputerEndianOrderAsString()).c_str());
}
return processed_val;
}
template <typename T>
T get_value_with_big_endianness(T val)
{
T processed_val = val;
if(GetThisComputerEndianOrder() == HL_LITTLE_ENDIAN)
{
SwapEndianness(processed_val);
}
else if(GetThisComputerEndianOrder() == HL_PDP_ENDIAN ||
GetThisComputerEndianOrder() == HL_UNKNOWN_ENDIAN)
{
throw std::runtime_error(std::string(std::string("This computer has unsupported endianness: ") + GetThisComputerEndianOrderAsString()).c_str());
}
return processed_val;
}
#endif // PATHTOOLS_H
#ifndef LOCKGUARD_H
#define LOCKGUARD_H
#include "pthread.h"
/**
* @brief The LockGuard class
* Protects against deadlock by forcing unlocks when going out of scope
*
* The mechanism is quite simple. Constructor locks, destructor unlocks,
* and custom locks and unlocks are possible
*/
class LockGuard
{
pthread_mutex_t& _lock;
bool is_locked;
public:
LockGuard(pthread_mutex_t& LockRef);
~LockGuard();
void lock();
void unlock();
LockGuard(LockGuard const &) = delete;
void operator=(LockGuard &) = delete;
};
#endif // LOCKGUARD_H
#ifndef SOUNDCONVERSION_H
#define SOUNDCONVERSION_H
#define WAVE_FORMAT_UNKNOWN 0x0000 /* Unknown Format */
#ifndef WAVE_FORMAT_PCM
#define WAVE_FORMAT_PCM 0x0001 /* PCM */
#endif
#define WAVE_FORMAT_ADPCM 0x0002 /* Microsoft ADPCM Format */
#define WAVE_FORMAT_IEEE_FLOAT 0x0003 /* IEEE Float */
#define WAVE_FORMAT_VSELP 0x0004 /* Compaq Computer's VSELP */
#define WAVE_FORMAT_IBM_CSVD 0x0005 /* IBM CVSD */
#define WAVE_FORMAT_ALAW 0x0006 /* ALAW */
#define WAVE_FORMAT_MULAW 0x0007 /* MULAW */
#include <iostream>
#include <string>
#include <regex>
#include <cmath>
#include "LockGuard.h"
#include "pthread.h"
#include "lame.h"
#include "WAVHeader.h"
extern pthread_mutex_t cout_mutex;
/**
* @brief ConvertFile
* Converts the file given by the path in the first parameter from WAV to MP3
* The file's header is read and cast to structs that extract information from the file
* Supported file types are PCM files of 8/12/16/24/32 bits and 32/64 floating point numbers, both mono and stereo
* @param filenamePtr
* file name as null terminated string
* @return
* no return value, just a NULL pointer
*/
void* ConvertFile(void* filenamePtr);
int GetElementSize(const WAVFormatHeader& header);
#endif // SOUNDCONVERSION_H
#ifndef THREADPOOL_H
#define THREADPOOL_H
#include <thread>
#include <queue>
#include <vector>
#include <functional>
#include <string>
#include <iostream>
#include "pthread.h"
#include "LockGuard.h"
/**
* @brief The ThreadPool class
* A basic thread pool implementation using pthread
*
* Call with the constructor ThreadPool::ThreadPool(N) to get N threads.
* The default constructor will detect the number of available cores/core threads and use it.
* Constructor throws std::runtime_error() on failure to spawn processes.
*
*
* Add tasks using ThreadPool::push_task(function, args). More info at the function description.
*
* Use ThreadPool::finish() to wait for all tasks to process and join all threads.
*
*
*
*
*/
class ThreadPool
{
pthread_cond_t task_queue_cond; //signaled when a new task is pushed
pthread_cond_t thread_finished_cond; //signaled when a thread finishes and no more is there to do
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
bool no_more_to_push = false;
typedef void*(*FunctionType)(void*);
//queue of tasks as an std::queue of std::pair of functions and args
std::queue<std::pair<FunctionType,void*> > tasks_queue;
void launch_threads();
int num_of_threads; //stores the number of threads requested
std::vector<pthread_t> threads_objects;
friend void* _thread_worker(void*);
bool joined = true; //prevents rejoining twice, true when all threads are joined
long threads_running = 0; //keeps track of threads that are executing tasks
void join_all();
public:
ThreadPool(int threads_count = (std::thread::hardware_concurrency() > 0 ? std::thread::hardware_concurrency() : 1));
virtual ~ThreadPool();
/**
* @brief push_task
* Adds a task to the processing queue
*
* Results are passed through parameters too. No return value is considered.
*
* @param function
* Function pointer to the function to be executed.
* The function is of type void*(void*)
* All parameters should be passed through a single void pointer
*
* @param params
* Parameters of the function as void*.
*/
void push_task(FunctionType function, void* params);
/**
* @brief finish
* Waits for all threads to finish and joins them.
*/
void finish();
};
#endif // THREADPOOL_H
#ifndef WAVSPECS_H
#define WAVSPECS_H
#include <cstdint>
#include <string>
#include <stdexcept>
#include <string.h>
#include "FilesTools.h"
/**
* All the below structs are to be mapped to parts of a WAV file header
* the getter functions help in detecting endianness of the data to read it correctly
*/
struct WAVFirstHeader
{
private:
/* RIFF Chunk Descriptor */
char RIFF[4]; // RIFF Header Magic header
uint32_t FileSize; // File Size - 8
char WAVE[4]; // WAVE Header
public:
bool check_RIFF_header() const;
uint16_t get_file_size() const;
bool check_WAVE_header() const;
};
struct WAVFormatHeader
{
private:
/* "fmt " sub-chunk */
char fmt[4]; // FMT header
uint32_t Subchunk1Size; // Size of the fmt chunk
// Audio format 1=PCM,6=mulaw,7=alaw,257=IBM Mu-Law, 258=IBM A-Law, 259=ADPCM
//A full list can be found online
uint16_t AudioFormat;
uint16_t NumOfChannels; // Number of channels 1=Mono 2=Stereo
uint32_t SampleRate; // Sampling Frequency in Hz
uint32_t byteRate; // SampleRate * NumChannels * BitsPerSample/8
// NumChannels * BitsPerSample/8 The number of bytes for one sample including all channels.
uint16_t blockAlign;
uint16_t bitsPerSample; // Number of bits per sample
public:
bool check_fmt_header() const;
std::string get_fmt_string() const;
uint16_t get_format_chunk_size() const;
uint16_t get_audio_format() const;
uint16_t get_num_of_channels() const;
uint32_t get_sample_rate() const;
uint32_t get_byte_rate() const;
uint16_t get_block_align() const;
uint16_t get_num_bits_per_sample() const;
void print(std::ostream& os) const;
};
struct WAVDataHeader
{
private:
/* "data" sub-chunk */
char Subchunk2ID[4]; // "data" string
uint32_t Subchunk2Size; // Sampled data length
public:
std::string get_data_string_before_data() const;
uint32_t get_data_chunk_size() const;
};
#endif // WAVSPECS_H
#ifdef _MSC_VER
#if _MSC_VER < 1900
#error This code was not tested to compile in Visual Studio versions older than 2015. Comment this line to try with earlier versions.
#endif
#endif
#if defined(_WIN32) || defined(__WIN32__) || defined(__TOS_WIN__)
#ifndef WIN32
#define WIN32
#endif
#endif
#include <iostream>
#include "ThreadPool.h"
#include "FilesTools.h"
#include "SoundConversion.h"
int main(int argc, char *argv[])
{
std::string target;
std::cout << "WAV to MP3 file converter by Samer Afach" << std::endl;
if(argc < 2)
{
std::cout << "Error: First argument should be the path where WAV files are located." << std::endl;
std::exit(1);
}
else
{
target = argv[1];
}
std::cout<<std::endl;
if(!IsValidDir(target))
{
std::cerr<<"Error: Directory path " << target << " is invalid. Exiting..." << std::endl;
std::exit(1);
}
//Get list of WAV files in the directory
const std::vector<std::string>& listOfFiles = GetFilesInDirectory(target,"^.*\\.([Ww][Aa][Vv])$");
if(listOfFiles.size() == 0)
{
std::cout<<"No files to convert were found in " + target << std::endl;
std::cout<<"Exiting."<<std::endl;
std::exit(0);
}
std::cout<<"List of files to be converted: " << std::endl;
for(auto it = listOfFiles.begin(); it != listOfFiles.end(); it++)
{
std::cout << *it << std::endl;
}
try
{
ThreadPool pool;
for(unsigned i = 0; i < listOfFiles.size(); i++)
{
pool.push_task(ConvertFile,(void*)listOfFiles[i].c_str());
}
pool.finish();
}
catch(std::exception &ex)
{
std::cout<<"An exception was thrown while processing data. Exception says: " << ex.what() << std::endl;
std::exit(1);
}
return 0;
}
#include "FilesTools.h"
long long FileSize(const std::string& filename)
{
std::ifstream in(filename.c_str(), std::ifstream::ate | std::ifstream::binary);
if(in.good() && in.is_open())
{
return in.tellg();
}
else
{
return -1;
}
}
char GetDirSeparator()
{
#if defined(WIN32)
return '\\';
#else
return '/';
#endif
}
void AddTailSlashToPath(std::string& path)
{
if(path.size() > 0 && path.back() != GetDirSeparator())
{
path += GetDirSeparator();
}
}
void FixPathSlashes(std::string& path)
{
#ifdef WIN32
std::replace( path.begin(), path.end(), '/', '\\');
#else
std::replace( path.begin(), path.end(), '\\', '/');
#endif
}
std::string JoinPath(const std::initializer_list<std::string>& params)
{
std::string output;
for(auto it = params.begin(); it != params.end(); it++)
{
AddTailSlashToPath(output);
output += *it;
}
return output;
}
bool IsValidFile(const std::string& path)
{
std::ifstream in(path.c_str(), std::ifstream::ate | std::ifstream::binary);
return (in.good() && in.is_open());
}
std::vector<std::string> GetFilesInDirectory(const std::string& path_p, const std::string& regex_filter)
{
//This is a cross-platform implementation of this function
//In reality I would've used boost::filesystem or Qt to handle this
//This can still be further improved to handle non-ASCII filenames
std::string path = path_p;
FixPathSlashes(path);
AddTailSlashToPath(path);
std::vector<std::string> listOfFiles;
#ifdef _MSC_VER
WIN32_FIND_DATA search_data;
memset(&search_data, 0, sizeof(WIN32_FIND_DATA));
std::wstring path_w(path.begin(),path.end());
//iterate over files in a directory by looking for X:\path\to\files\*.*
HANDLE handle = FindFirstFile((path_w + L"*.*").c_str(), &search_data);
while(handle != INVALID_HANDLE_VALUE)
{
//ignoring non-ascii names as this is just an example
//bare filename (without full path)
std::string filename_bare = std::string(CW2A(search_data.cFileName));
//filename with full path
std::string filename = JoinPath({path,filename_bare});
if(IsValidFile(filename) && std::regex_match(filename,std::regex(regex_filter.c_str())))
{
listOfFiles.push_back(std::string(filename));
}
if(FindNextFile(handle, &search_data) == FALSE)
break;
}
FindClose(handle);
return listOfFiles;
#else
DIR *dpdf;
struct dirent *epdf;
dpdf = opendir(path.c_str());
if (dpdf != NULL)
{
while(true)
{
if(!(epdf = readdir(dpdf)))
{
break;
}
std::string filename = JoinPath({path,epdf->d_name});
if(IsValidFile(filename) && std::regex_match(filename,std::regex(regex_filter.c_str())))