Welcome to AIFS Documentation

This project is hosted by Digital Oversight.

The code is open source, and available on Github.

About AIFS

AIFS v1.02

Jan 9th, 2016

This framework implement large scale information gathering and analysis in order to provide valuable intelligence to a third party software. Using multiple source, this software calculate the risk factors related to a decision, regardeless of the initial objective and the resulting actions.

AIFS tries to implement general patterns related to machine learning and intelligence classification. The core functionalities include libraries related to public content information, digital network information, geographical and human information.

Note that the code forming initial Github release of AIFS is a collection of tools and projects initially developed during 2006 and 2011. Since many classes and functions are now aged, you will notice many parts of the source follow php4 styles and standards. The following developers participated in the early unreleased version of this software (pre 2012) : Vincent Menard, Julien Jouvent-Halle, Rabih Majzoub, Jean-Fran?ois Giroux, Daniel Greenberg.

License

Copyright (c) 2016, Digital Oversight All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

  1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
  2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

The views and conclusions contained in the software and documentation are those of the authors and should not be interpreted as representing official policies, either expressed or implied, of the FreeBSD Project.

Downloads

How to install

# git clone https://github.com/digitaloversight/aifs.git

# mysql -u root -p < ./aifs/schema/aifs-base-1.02.sql

mysql> insert into user set Host='localhost', User='aifs', Password=PASSWORD('');

mysql> flush privileges;

mysql> grant all on aifs.*  to 'aifs'@'localhost';

Configuration

Make sure you reviwed the configuration on the following files

``/config/config.php``

``/config/tool/DomainSelector.php``

``/common/sql/Sql.php``

Extends

AIFS comes with extensions, please take a look at the aifs-extends repository available on Github.

Configuration

If you do not load the web administration panel, you will need two things to enable basic features of AIFS : a keyword dictionary and an url list.

Keyword dictionary

If you wish to enable semantic indexing, you need to load a dictionary. We provide the English-French words list in the aifs-extends repository. Load it on a freshly installed aifs-base-1.02.sql schema using this command :

# git clone https://github.com/digitaloversight/aifs-extends.git

# tar -xvzf ./aifs-extends/data/osint_keyword_EN_FR_102.sql.tar.gz

# mysql -u root -p aifs < ./aifs-extends/data/osint_keyword_EN_FR_102.sql

URL list

If you do not use the web administration panel to provide urls, you will need to fill the osint_url manually.

mysql> insert into osint_url set url='https://twitter.com/infainit';

Routines

OSINT routines

Fetch Version routine/osint_fetch_version.php

Perform a version check with a randomly choosen url in the user filled url table.

Changes routine/osint_changes.php

Execute a comparison algorithm between two saved version of a document stored.

Title routine/osint_title.php

Extract title from html content.

Tag count routine/osint_tagcount.php

Precalculate the number of urls linked for all tags stored in the osint_tags

Table Rotation routine/osint_table_rotation.php

Rotate the web content present into the main version content by renaming the sql table.

DNINT routines

Semantic Analysis routine/dnint_url_semantic.php

Apply semantic analysis against html parsed content.

Oubound routine/dnint_oubound.php

Get the outbound links on an html version.

NS Check routine/dnint_ns_check.php

Perform an NS check on a saved url.

Google PR routine/dnint_google_pr.php

Fetch the google pr for a specified url.

AIFS ocr

This software includes a basic OCR (optical caracter reader) binary designed to parse visible text from image files. As of version 1.02, the ocr feature is not linked with the semantic indexer nor with any image feeder available in the aifs-extends package.

Requirements

OpenCV-2.4.0

tesseract-3.01

leptonica-1.69

Builds are made are available for debian linux machines.