Abstract:
Introduction/purpose: This paper reports on a pilot comparative analysis of the Cuckoo and Drakvuf sandboxes. These sandboxes are selected as the subjects of the analysis because of their popularity in the professional community and their complementary approaches to analyzing malware behavior. Methods: Both sandboxes were set up with basic configurations and confronted with the same set of malware samples. The evaluation was primarily conducted with respect to the question of to what extent a sandbox is helpful to the human analyst in malware analysis. Thus, only the information available in Web console reports was considered. Results: Drakvuf is expected to perform better when confronted with evasive malware and so-called “file-less” malware. Although still not mature in terms of integration, customization and tools, this sandbox is considered a second generation sandbox because of its agentless design. On the other hand, the Cuckoo sandbox creates a better overall experience: it is supported through good documentation and strong professional community, better integrated with various tools, support more virtualization, operating system and sample types, and generates more informative reports. Even with a smaller capacity to prevent evasive malware, its Python 2 agent script makes it more powerful than Drakvuf. Conclusion: To achieve the optimal open-source sandbox-based protection, it is recommended to apply both the Cuckoo and Drakvuf sandboxes. In circumstances of limited resources, applying the Cuckoo sandbox is preferable, especially if exposure to malware deploying evading techniques is not frequently expected.
Keywords: Sandbox, Cuckoo, Drakvuf, Malware behavior analysis.
Pезюме:
Введение/цель: В данной статье представлен экспериментальный сравнительный анализ программных сред песочниц Cuckoo и Drakvuf. Эти системы были выбраны в качестве предмета анализа из-за их популярности в профессиональном сообществе и их взаимодополняющих подходов к анализу воздействия вредоносных программ. Методы: Обе системы имеют базовые настройки и подвергаются воздействию одного и того же набора вредоносных программ. Анализ преимущественно проводился с целью выявления степени полезности песочниц для аналитика-человека при анализе вредоносных программ. Следовательно, учитывалась только та информация, которая была доступа в отчетах веб-интерфейсов наблюдаемых систем. Результаты: Можно ожидать, что Drakvuf даст лучшие результаты при воздействии вредоносных программ, использующих методы обхода песочниц в виртуальных средах. Несмотря на то, что данная среда пока не достигла своей полной мощности относительно интеграции, настроек и доступных программных инструментов, ее все-таки можно считать представителем второго поколения изолированных систем программной среды, благодаря ее безагентной технологии. С другой стороны, песочница Cuckoo в целом более удобна для пользователей: она поддерживается хорошей документацией и сильным профессиональным сообществом, лучше интегрирована с различными программными инструментами, поддерживает больше видов виртуализации, типов операционных систем и образцов, в том числе она лучше генерирует отчеты. Несмотря на то, что у этой песочницы гораздо меньше возможностей предотвращения атак вредоносных программ в виртуальной среде, применение сценария с выявлением действий вредоносных программ делает эту песочницу более эффективной. Выводы: Для достижения оптимальной защиты на основе песочницы с открытым исходным кодом рекомендуется применять как песочницы Cuckoo, так и Drakvuf. В условиях ограниченных ресурсов предпочтительнее применять песочницу Cuckoo, особенно если не предполагается частое воздействие вредоносных программ, использующих метод уклонения от обнаружения.
Ключевые слова: изолированный запуск программы, Cuckoo, Drakvuf, динамический анализ вредоносных программ.
Abstract:
Увод/циљ: У раду се приказује упоредна пилот-анализа софтверских окружења Куку и Драквуф за изоловано извршавање програма. Ови системи одабрани су за предмет анализе због своје популарности у професионалној заједници и комплементарних приступа анализи понашања злонамерних програма. Методе: Оба система постављена су на основна подешавања и изложена истом скупу злонамерних програма. Анализа је примарно урађена с аспекта процене степена информативности добијених извештаја о извршавању злонамерних програма за људског аналитичара. Стога су, као предмет анализе, узете у обзир само информације доступне у веб-интерфејсима посматраних система. Резултати: Може се очекивати да ће Драквуф остварити бољи учинак када се изложи злонамерним програмима који примењују технике избегавања извршавања у виртуалним окружењима. Иако ово окружење још увек није остварило пун капацитет у смислу интегрисања, прилагођавања и доступних софтверских алата, може се сматрати представником друге генерације система за изоловано извршавање програма, због свог дизајна који искључује примену софтверског агента. С друге стране, окружење Куку ствара боље свеукупно корисничко искуство: подржано је добром документацијом и јаком професионалном заједницом, боље је интегрисано са различитим софтверским алатима, подржава више типова виртуелизације, оперативних система и типова узорака и генерише информативније извештаје. Иако поседује мањи капацитет за откривање злонамерних програма који примењују технике избегавања извршавања у виртуалним окружењима, могућност примене скрипти с дефиницијама злонамерног понашања програма чини ово окружење ефективнијим. Закључак: Да би се постигла оптимална заштита, заснована на окружењима отвореног кода за изоловано извршавање програма, препоручује се примена оба разматрана система. У условима ограничених ресурса, примена система Куку пожељнија је, посебно ако се не очекује често излагање злонамерним програмима који примењују технике избегавања извршавања у виртуалним окружењима.
Keywords: изоловано извршавање програма, Cuckoo, Drakvuf, динамичка анализа злонамерних програма.
Original scientific papers
A pilot comparative analysis of the Cuckoo and Drakvuf sandboxes: An end-user perspective
Экспериментальный сравнительный анализ песочниц Cuckoo и Drakvuf: взгляд конечного пользователя
Упоредна пилот-анализа софтверских окружења Куку и Драквуф за изоловано извршавање програма: перспектива крајњег корисника
Received: 01 February 2022
Revised document received: 16 March 2022
Accepted: 18 March 2022
The number of malware attacks has recently increased significantly (e.g., it has been doubled in the period between 2015 and 2019 (Melvin & Kathrine, 2020) and the average time needed to detect a data breach is considerable (e.g., in 2020 it took 203 days in average), (IBM, 2020). The identification of highly sophisticated, target specific and stealthy operated cyber threats is a challenging task, because of their underlying characteristics such as encrypted covert communication, sophisticated attack techniques, continuous monitoring and control of victim’s resources, wiping or masking the traces, etc. (Chakkaravarthy et al, 2019)
Due to the complexity and severity of advanced cyber threats, defenders of valuable assets aim at discovering threats before they get in a defensive perimeter. In line with this aim, this paper provides a pilot comparative analysis of two open-source and the most frequently used sandbox solutions: Cuckoo and Drakvuf.
A cybersecurity sandbox is a physical or virtual environment used to execute suspicious file samples or run programs without interfering with a monitoring system or permanently affecting a device they are running on (Arntz, 2020; Chakkaravarthy et al, 2019). The sandboxing is used to detect potentially malicious codes and applications before serving them up to critical devices (Arntz, 2020). The detection is based on malware behavior analysis, which may be roughly described by an analogy to biometric behavioral description (Tot et al, 2021).
A sandbox usually consists of a management part and virtual machines (VMs) which represent victim hosts. VMs are typically configured similarly to virtual and physical computers in a given organization in order to mimic the production environment which is being protected from malware attacks. When suspected files are executed in these VMs, it is possible to monitor their behavior and react before they occur in a production environment.
The Cuckoo and Drakvuf sandboxes are selected as the subjects of the analysis in this study because of their popularity in the professional community and their complementary approaches to analyzing malware behavior. Cuckoo uses a Python script-shaped agent in the analysis VM, while Drakvuf applies an agentless approach. The network architecture adopted in the reported study is shown in Figure 1.
Both sandbox environments can be accessed from the management network for the purposes of configuration and submission of samples for analysis through a Web interface. The characteristics of individual VMs are provided in Table1.
It is usual for Cuckoo and DRAKVUF to be installed on bare-metal (or in VM) and to have analysis VMs nested (i.e., Type 2 virtualization). In this study, we applied a different approach for Cuckoo since it supports VMware ESXi API calls. The ESXi communication with VMs allows for a more configurable environment and a more comprehensive analysis (e.g., by taking a snapshot and applying additional tools outside a sandbox environment). In contrast, this approach is not feasible for Drakvuf, so the nested Xen virtualization is applied as the underlying hypervisor.
The pfSense firewall is configured in front of the whole environment in order to:
- prevent any traffic from leaving the experimental environment,
- provide additional real-time monitoring of network connections induced by the analyzed samples, and
- allow for keeping track of the sandbox-based traffic analysis across time.
A basic insight into the design of the sandboxes is provided below.
The Cuckoo sandbox allows for dynamic detecting of runtime behaviors in an isolated environment, i.e. a virtual machine (including API calls, network traffic, files dropped, etc.) by the use of signatures, written as Python 2 scripts, that detect a broad range of malware, from a simple key logger to a more complicated execution of a process that has an injected code.
The malware detection is achieved via cuckoomon.dll, a dynamic link library injected into a process that allows for run-time logging of its behavior manifestations, which are then reported back to the main Cuckoo sandbox process.
Cuckoo may be integrated with local email solutions and intrusion prevention systems to identify ransomware and other potentially malicious entities, and to prevent potential breaches and data loss.
The analyst uses a Cuckoo host machine to manage the analysis through a command line or available Web interface. In the scope of an analysis, malware samples are submitted and reports are collected (Figure 2). Before malware execution, a VM’s snapshot is reversed to the initial state which ensures that tracks of previous analyses do not interfere with the currently ongoing analysis. An in-guest Python agent serves to execute a malware sample and send a report back to the Cuckoo host.
Drakvuf (Lengyel et al, 2014) is a VMI-based sandbox which has the ability to trace kernel-level and user-level malware (Melvin & Kathrine, 2020). VMI stands for Virtual Machine Introspection – external access to the virtual hardware state, which can monitor process execution, file operations, system calls and kernel function traces, all at the hypervisor level, with the ability to spot kernel rootkits and reduce the possibility for malware to use its evading techniques (Melvin & Kathrine, 2020). Instead of an in-guest agent, Drakvuf uses the breakpoint injection technique in which an instruction is written into the VM’s memory at code locations of interest (Lengyel et al, 2014). By configuring a CPU to issue VMEXIT when breakpoints are executed, and configuring Xen (a virtualization hypervisor) to forward these events to control domain, DRAKVUF is capable of trapping the execution of any code within the analysis machine. The #BP technique, previously used for the stealth debugging, is used for automatic execution tracing of the entire operating system, including also the Kernel internal functions.
Drakvuf’s initial tests showed a great potential and the development of this sandbox was continued, providing a modern and powerful open-source malware analysis platform. The project is maintained and available at: https://drakvuf.com. In this study, we use the customized solution called Drakvuf sandbox (CERT of Poland) which is an actively developed project (Cert Polska, 2021;2022), since it is easy to install and configure.
Since the considered sandboxes are not of the same type, it is challenging to introduce a metric for their comparison. Instead, we decided to use a descriptive approach to evaluate different features we consider relevant. The observed features are shown in Table 2, where sign “+” is assigned to the sandbox that performs better with respect to a given feature.
Both sandboxes are available for free under the General Public License (GNU GPLv3) and reasonably well documented. As a more mature solution, Cuckoo is more extensively documented, which allows for comparatively easier installation and configuration. In addition, this sandbox is used by many organizations, including CERT of Poland, CERT of Estonia (publicly available at https://cuckoo.cert.ee.), Checkpoint, Avira, etc. (Estonian Information System Authority - RIA, 2017; Checkpoint Software Technologies LTD, 2019; Sick, 2014; CERT Polska, 2019), and the professional community is strong in terms of use, problem solving, customization and modifications. However, since Cuckoo is based on Python 2 (version 2.7 is used in this study), it suffers from some problems caused by dependencies on Python 3 packages. This fact prevented us from using the latest 2.07 Cuckoo version in this study, and led us to downgrade the hosting operating system from Ubuntu 20.04 to Ubuntu 18.04. The Drakvuf sandbox has good basic documentation and respectable community. The Drakvuf Sandbox (CERT of Poland edition) is enriched with a Web interface and additional plugins which create the user experience near to Cuckoo. However, with respect to the complexity of installation and setup, Drakvuf performs better, mainly due to the packet dependency problems in Cuckoo. A full rewrite of Cuckoo for Python 3 in cooperation with the CERT of Estonia is announced and its availability for customization is expected to improve the Cuckoo’s rating with respect to this feature.
Although there are certain modifications of Cuckoo aimed at achieving the scalability for the Amazon Web Services (AWS Cloud), this sandbox is generally not easy scalable. In contrast to this, Drakvuf is easily scaled (i.e., command “draksetup scale n” accepts an input argument that represents the number of instances to be automatically configured and started for parallel samples execution).
Table 3 provides a comparative overview of the content provided in the analysis reports of the Cuckoo and Drakvuf sandboxes.
It could be observed that Drakvuf lacks many report features compared to Cuckoo, some of which could be derived from log files. The log files generated by both sandboxes are rather informative but not necessarily appropriate for human interpretation, and therefore we consider here only the information available in Web console reports.
However, the reporting functionality is primarily evaluated with respect to the question of to what extent a sandbox is helpful to the human analyst during the process of malware analysis. To assess this question, we have used 21 potential malware files, summarized in Table 4.
The samples are courtesy of the Virus total portal (Sood, 2021). The name of each sample (cf. the second column) is generated by taking the first 8 characters of its SHA-256 value.
The full 256-bit hashes are available but not provided because of possible misuse. The fourth column of Table 4 contains the numbers of antivirus engines that reported the sample as positive, while the fifth column contains the numbers of all antivirus engines that analyzed the sample. We introduce the following malware score to describe a sample (note that malware score will always be in range 0 to 1) as a division of number of positives (#P) by total number (#T).
The malware score of a sample is used as an external measure according to which the respective behavior reports obtained from the Cuckoo and Drakvuf sandboxes were evaluated. The details of this evaluation are given in Table 5.
Although Cuckoo had some difficulties analyzing malware samples that operate on a large number of files (i.e., too many files error) due to which multiple analysis restarts were required and the sandbox failed to produce reports for two samples in Table 5 (samples 002d7712 and 003add9c), the overall conclusion is that it provides more informative reports.
From Table 3, it can be observed that Cuckoo is more efficient for the given data. However, it should be noted that Drakvuf configuration supposes a constant execution time (default is 10 minutes, but could be lowered). The Drakvuf authors probably introduced this unbalanced trade-off between the efficacy and security in order to reduce the possibility for a sample to evade the sandbox environment.
The file types supported by the sandboxes are shown in Figure 4. The Cuckoo sandbox has a huge advantage in terms of supported file types (including various scripts, PDF and ZIP-file type extensions) and can be customized for generic packages by selecting applications to handle a particular sample type.
Since malware proved as evasive was not available in this study, the insight with respect to this feature is based on the sandboxes design and available research (Laing, 2017; Mills & Legg, 2021; Ferrand, 2015; Lengyel et al, 2014). It may be concluded that out-of-the-box Drakvuf performs better when confronted with evading malware, although both systems could be hardened to increase the probability of executing and reporting evasive malware.
Cuckoo supports Windows XP, Windows 7 64-bit and Windows 10 64-bit (not fully functional), Ubuntu 18.04 as a Linux guest operating system, and it can be configured to analyze samples in an Android environment under the Linux guest operating system. The configuration of Physical machine is also possible for the purpose of analysis. Drakvuf supports Windows 7-8, both 32-bit and 64-bit versions, 64-bit version of Windows 10 as well as 32-bit and 64-bit Linux systems running kernel 2.6.x and above, while the particular Drakvuf sandbox considered in this study is limited to Windows 7 (64-bit) and Windows 10 (64-bit) experimental.
Due to its design, Drakvuf is limited to the Intel processors, while Cuckoo can run without hardware limitations. The hypervisor support is also on the side of Cuckoo because it can communicate with analysis VMs under Xen, KVM, VMware ESXi, Oracle Virtual box, and almost on any other platform, while in the Drakvuf environment only Xen is natively supported (KVM is in the experimental phase and VMware Workstation needs to be additionally configured). Due to its design, Drakvuf only supports nested virtualization. Thus, Cuckoo is also more advanced with respect to this feature, although Drakvuf provides enough options to work just fine in most environments.
Since Cuckoo is a more mature solution, its integration possibilities are greater. Thus, it can be integrated with a range of tools for additional analysis (e.g., Cuckoo-droid, Signature updates, YARA rules, Suricata, Snort, Moloch, Volatility, Virustotal integration, etc) (Ashby, 2015; Checkpoint Software Technologies LTD, 2015). The Drakvuf sandbox allows integration with certain tools (e.g., Volatility and procmon for behavioral graph induction), but its integration possibilities are still significantly lower.
Cuckoo supports multiple samples submission, which in conjunction with its efficient execution allows a real-time analysis in environments with limited resources and may be applied to analyze large amounts of malware. REST API is implemented and easily accessible by the execution of a single command, enabling the automation of the analysis process. Drakvuf does support multiple samples submission but its API is still undocumented and we could not find the way to effectively use it.
Signatures are probably the most lacking feature in Drakvuf. In contrast, Python scripts in Cuckoo are automatically updated from the repository and create signatures that recognize malicious behavior of samples. YARA rules can be defined and applied to improve this process. Signatures are also applied in the static analysis of samples.
Cuckoo has a beta version scoring system which is visually very illustrative, but not fully informative for detailed analyses in which visually advanced reports with signatures are substantially useful. In Drakvuf, the Process tree and Behavioral graph are very useful visual tools which make Drakvuf a slightly more advanced solution with respect to the visualization functionality.
Table 2 shows that the Cuckoo sandbox performs better with respect to many features. However, the answer to the question of which sandbox to apply depends on the expected malware behavior.
Drakvuf is expected to perform better when confronted with evasive malware and so-called “file-less” malware (residing only in the RAM of a device). It is also suitable for capturing traces that a malware attempts to clean (i.e., deletion of temporary files), since it fetches deleted files by intercepting internal kernel calls related to the file deletion operations. On the other hand, Drakvuf has its limitations including the use of the injection mechanism to automatically start a sample (Lengyel et al, 2014) which a malware can exploit to evade an abnormal start, but research demonstrates the potential of this sandbox with respect to evading malware techniques. Although still not mature in terms of integration, customization and tools, it is considered a second generation sandbox because of its agentless design (Laing, 2017; Richards, 2021; Lengyel et al, 2014)
The Cuckoo sandbox creates a better overall experience: it is supported through good documentation and strong professional community, better integrated with various tools, supports more virtualization, operating systems and sample types. With the Python 3 rewrite, Cuckoo 3 (Hatching International B.V., 2022) is expected to perform even better. Even with a smaller capacity to prevent a malware to evade the sandbox environment, its Python 2 agent script makes it more powerful than Drakvuf. Recent research including 539 organizations and companies in Europe and USA (Spiceworks, 2019) shows that 92% of the companies apply server virtualization solutions, and predicts that the increasing number of VMs in production environments could result in lowering the frequency of evasion techniques since attackers probably would not allow to be deprived of the opportunities to target these machines.
At the given point, to achieve an adequate or optimal open-source sandbox-based protection and improve cyber security risk management practices (Ilić, 2012), it is recommendable to apply both the Cuckoo and Drakvuf sandboxes. In circumstances of limited resources, applying the Cuckoo sandbox is preferable, especially if exposure to malware deploying evading techniques is not frequently expected.
FIELD: Computer sciences, IT, Cyber security
ARTICLE TYPE: Original scientific paper
https://scindeks.ceon.rs/article.aspx?artid=0042-84692202372I (html)
https://aseestant.ceon.rs/index.php/vtg/article/view/36196 (pdf)
https://doaj.org/article/601cb1811d5d4364b2853c27a82faa7b (pdf)
https://elibrary.ru/item.asp?id=48140914 (pdf)
http://www.vtg.mod.gov.rs/archive/2022/military-technical-courier-2-2022.pdf (pdf)
slavisa.ilic@mod.gov.rs
