By Thanya Kunakornpaiboonsiri | 20 March 2013
Big Data solution has played a key role in helping Thailand’s Department of Special Investigation (DSI), Ministry of Justice to gather and analyse massive amount of data to optimise its crime investigations.
The DSI implemented a Microsoft Big Data solution based on Microsoft SQL Server 2012 and Apache Hadoop software to improve investigation processes and reduce manual procedures.
“With the traditional approach, it took two years to search for tips and gather and analyse data. With the new solution, it took only 15 days,” said DSI Deputy Director-General Pol Col Yanaphon Youngyuen.
Established to handle major criminal investigations, the DIS is a national law enforcement agency dedicated to stopping and investigating specially assigned criminal activities and cases.
The agency’s large data set includes over one million records gathered from multiple sources in both structured and unstructured formats, such as image, videos, and documents.
“It was very difficult to mine through the data, and the results were too broad with unclear targets,” said Pol Col Youngyuen. “This often forced us to send personnel to the actual crime scenes, which cost us a lot of time and money.”
The DSI collaborates and shares information with other agencies to monitor suspected persons and transactions in real time. The track information includes telephone calls, financial transactions, and passports records, according to him.
However, to search data for important tips, officers needed to create specialised queries. Instead, the department wants “a system that could automatically notify of any suspicious persons or activities.”
Pol Col Youngyuen showed examples of such activities that “Like when there are many foreign criminals pouring into Thailand, and all travel to the same location, or when there is a noticeably large sum of money being transferred in the country.”
“If we had a system that notifies us about this, we could implement proactive measures to prevent crimes from happening,” he said.
In partnership with HP—who provided servers for testing, and Betimes Solutions that help to implement the Big Data Solution supplied by Microsoft, the DSI was able to import and analyse huge volumes of data from multiple sources.
For easier data mining, the solution also included Apache Hadoop software, an open-source platform that the DSI uses to store large volumes of unstructured data. Metadata is stored in the Hives data warehouse system.
The DSI is taking advantage of self-service business intelligence (BI) tools, and data management capabilities, and other feature to present reports to high-ranking officers that include analysis details and an executive summary of each case monitored with helpful visual features.
The department planned to apply the solution to current and future investigation. It will also further expand the use of the new solution and implement its own private cloud to manage the security of confidential data, added Pol Col Youngyuen.