Fuite de données a la SNCB: Géolocalison les entrées


*UPDATE*: les images ci-dessous peuvent etre reprises pour d'autres articles si une reference est gardee ;-)

Une fois n'est pas coutume, j'effectue la traduction d'un de mes articles en français, le public visé dans celui-ci étant principalement Belge.

Xavier Damman nous a montré quel genre de statistiques on pouvait extraire des données que la SNCB a accidentellement mis à disposition.

Mais que pouvons nous en apprendre en utilisant de la géolocalisation massive des adresses contenues dans ce fichier? Qu'est-ce que cela représente au niveau des individus présents dans ce fichier? Jusqu'ou pouvons-nous aller?

D'abord, examinons une vue mondiale de la géolocalisation des villes impactées (Cliquez pour agrandir):

World MAP repartition of SNCB Customers

Cette carte agrège les villes ou des clients de la SNCB ont été trouvés. Plus le point est gros, plus il ya de villes contenant des clients sous ce point.

Effrayant, non?

On peut aller un peu plus loin en se concentrant sur le pays qui nous intéresse le plus ici: La Belgique.

Géolocalisons cette fois les rues et plaçons les sur une carte de Belgique:

Belgium MAP repartition of SNCB Customers

Je ne divulgerai pas ici une vue plus précise, meme si l'obtenir serait tres facile. Il ne serait pas difficile d'ajouter une fonction afin d'obtenir les details personnels d'une personne via un simple clic. 

Serait-il réellement difficile de recouper ces informations avec des sites publics comme Facebook ou FourSquare? Combien de fuites de données comme celle-ci faudra-t-il avant que l'on commence à 'interesser à nos données privées?

La SNCB est certainement à blâmer dans le cas présent, mais ce n'est certainement pas la première fois ni la dernière fois qu'une si grande base de données est mise à disposition sur Internet! Nous devons preter attention à qui nous confions nos données privées ainsi que le fait les organismes qui les utilisent le fassent avec soin.

Follow me on twitter: @tgouverneur

SNCB Data Leak: Geo-localization of the entries


*UPDATE*: Images below can be used to build up other article if reference is kept back.

Xavier Damman already showed what can be extracted from the SNCB's leaked data. Now what could be shown with some massive geo-localization of theses data? What would it means to individuals present in the files? How far can we go?

First, let's examinate a world-view of each entry's town (Click to enlarge):

World MAP repartition of SNCB Customers

This map aggregate town where SNCB customers have been found inside the leaked file. The bigger the number is on the point, the more town have customers behind the point.

Scary, isn't it?

We can drill-down that view by focusing on the most used country: Belgium.

Let's Geo-localize streets this time and put them on a Belgium's map:

Belgium MAP repartition of SNCB Customers

I will of course not disclose a more drilled-down view, even if having such a view is easy. 

It would also not be hard to add a function to show the personal detail of each marker with a simple click.

How hard would it be to cross-link theses data to public websites like facebook or foursquare? How many leaks like that will we need to start worrying about our private data?

SNCB can be pointed out this time, but it's not the first leak and certainly not the last! We should all take care of where we put our private data and ensure that organism who handle them do it with care...

Follow me on twitter: @tgouverneur

SUNWjet: Add a slice 7 to a zpool’s disk

According to Oracle's documentation, if you want to use SUNWjet to jumpstart a server with a ZFS Root pool and a slice 7 to put metadb on, you must first parition your drives and then launch the jumpstart.

This is particularly annoying as two operations are required.

I've found this post while searching on Google to see if it was possible to create a slice 7 automatically during the jumpstart. Unfortunately, it didn't worked as:

  • Disks were hardcoded
  • Line 321 seemed to be architecture dependant (i86pc)

As I wanted to add this permanently and as every server I jumpstart can possibly one day use either UFS/DiskSuite or metaset, I wanted to have a slice 7 on every server in case of future use.

I wrote then this little patch to adapt the "populate_client_dir" script to add a slice 7 of 100Mb on every disk specified to be used as root pool:

So, to apply the patch, simply run:

 cd /opt/SUNWjet/Utils/solaris
patch -p0 < /tmp/solaris-populate_client_dir.patch

Then, run the make_client script against your template, where you would have specified the zpool spec:

base_config_profile_zfs_disk="c0t0d0s0 c1t0d0s0"

Solaris 11 ISC DHCP: Cannot specify multiple interfaces

While trying to configure the ISC DHCP server on Solaris 11 to serve my local VLANs, I wanted to restrict its usage to only three interfaces, I then issued the following setprop command:

# svccfg -s svc:/network/dhcp/server:ipv4 config/listen_ifnames astring\: "vlan100 vlan201 vlan202"
# svcadm refresh -s svc:/network/dhcp/server:ipv4
# svcadm enable svc:/network/dhcp/server:ipv4

But the service was failing to start... after adding some debug echo's to the /lib/svc/method/isc-dhcp file, I saw that the whitespaces of this property get escaped when retrieved from the method's script:

# tail -3 /var/svc/log/network-dhcp-server:ipv4.log
[ Jul 10 13:44:02 Executing start method ("/lib/svc/method/isc-dhcp"). ]
IFACES: vlan100\ vlan201\ vlan202
[ Jul 10 13:44:02 Method "start" exited with status 95. ]
# ggrep -B 4 IFACES /lib/svc/method/isc-dhcp
get_dhcpd_options() {
# get listen_ifname property value.
LISTENIFNAMES="`get_prop listen_ifnames`"
echo "IFACES: ${LISTENIFNAMES}";
# /usr/lib/inet/dhcpd -f -d -4 --no-pid  -cf /etc/inet/dhcpd4.conf -lf /var/db/isc-dhcp/dhcpd4.leases vlan100\ vlan201\ vlan202
vlan100 vlan201 vlan202: interface name too long (is 23)
# /usr/lib/inet/dhcpd -f -d -4 --no-pid  -cf /etc/inet/dhcpd4.conf -lf /var/db/isc-dhcp/dhcpd4.leases vlan100 vlan201 vlan202
Internet Systems Consortium DHCP Server 4.1-ESV-R4

So, to fix this behaviour, edit the /lib/svc/method/isc-dhcp, line 66 should be changed:

LISTENIFNAMES="`get_prop listen_ifnames`"

by

LISTENIFNAMES="`get_prop listen_ifnames|sed -e 's/,/ /g'`"

Then, you can set the listen_ifnames properties with multiple interfaces separated by commas:

# svccfg -s svc:/network/dhcp/server:ipv4
svc:/network/dhcp/server:ipv4> setprop config/listen_ifnames = astring: "vlan100,vlan201,vlan202"
svc:/network/dhcp/server:ipv4> exit
# svcadm refresh svc:/network/dhcp/server:ipv4
# svcadm disable svc:/network/dhcp/server:ipv4
# svcadm enable svc:/network/dhcp/server:ipv4

WeSunSolve: One year later

More than one year ago, the WeSunSolve website has been launched publicly to address the lack of information available for the Solaris operating system.

Facing it's success, a lot of improvements, features and stuff were added.. and visitors keeps like it!

Here are some statistics about this past year on the website:

Visitors

  • 307191 Unique Visits
  • 657270 Page viewed
  • 5500 Downloads
  • 64% Bouncing Visits
  • 670 Registered Users

Countries

  • 1. United States
  • 2. United Kingdom
  • 3. Germany
  • 4. Japan
  • 5. France

Website

  • Number of patches registered: 75928
  • Number of readmes version gathered: 63579
  • Number of checksums registered: 59471
  • Number of BugIDs registered: 365191
  • Total size of the patches repository: 623.57 GBytes
  • Number of Files detected: 1323021
  • Number of Packages: 8639
  • Number of CVE: 438

I would like to thank all the people who have made bugs reports, features requests and comments as well as the ones who have simply put their thumbs up!

If you like WeSunSolve, please spread the word! Talk 'bout it with your colleagues and share your experiences! If you're achieving a recurrent task using WeSunSolve, why not writing a little Howto?

You're part of a team of Solaris sysadmin? Did you know that you can know work in collaboration with your colleague on WeSunSolve? Check the documentation for more information ;-)

Last but not least, do not hesitate to send me your thoughts on the website! It's always good to hear from people who are using your work :) Especially when it's free ;-)

Sending PGP HTML Encrypted e-mail with PHP

While adding the PGP HTML Report feature to WeSunSolve, I first successfully crypted the content of the HTML report to be sent to the user with PGP key. I would have thought that this was gonna be the hardest part, that was without thinking about MIME and HTML support of PGP encrypted mails.

Here is how I finally ended up by creating HTML PGP encrypted Mails using PHP which can be opened using (at least) claws-mail and thunderbird with proper rendering of the HTML report:

Content of the clear message

To: test@test.com
Subject: My HTML crypted report
X-PHP-Originating-Script: 1000:mlist.obj.php
From: "We Sun Solve" 
Reply-to: admin@wesunsolve.net
X-Sender: WeSunSolve v2.0
Message-ID: <1335717276@wesunsolve.net>
Date: Sun, 29 Apr 2012 18:34:36 +0200
MIME-Version: 1.0
Content-Type: multipart/encrypted;
 protocol="application/pgp-encrypted";
 boundary="------------enig029BFFF948226050D5D90E10F"

This is an OpenPGP/MIME encrypted message (RFC 2440 and 3156)
--------------enig029BFFF948226050D5D90E10F
Content-Type: application/pgp-encrypted
Content-Description: PGP/MIME version identification

Version: 1

--------------enig029BFFF948226050D5D90E10F
Content-Type: application/octet-stream; name="encrypted.asc"
Content-Description: OpenPGP encrypted message
Content-Disposition: inline; filename="encrypted.asc"

-----BEGIN PGP MESSAGE-----
Version: GnuPG v1.4.10 (GNU/Linux)

****SNIPPED PGP CRYPTED BASE64 MESSAGE ****
-----END PGP MESSAGE-----

--------------enig029BFFF948226050D5D90E10F--

Content of the PGP encrypted part

Content-Type: multipart/alternative;
 boundary="------------F983FADF500537B8AFDC5E483"

This is a multi-part message in MIME format.
--------------F983FADF500537B8AFDC5E483
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

You need to have a MUA capable of rendering HTML to read
the WeSunSolve emails.

You can consult the website http://wesunsolve.net if you
are not able to read this email, the information sent to you
should also be on the website...

--------------F983FADF500537B8AFDC5E483
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

This is the report in cleartext!

--------------F983FADF500537B8AFDC5E483--

Code Used

$pgpmime = '';
$mime = '';
$headers = '';
$dest = 'test@test.com';
$subject = 'My HTML crypted report';
$clearContent = '<html><p>This is the report in cleartext!</p></html>';
$clearText = 'This is the text version of the report';
/* Prepare the crypted Part of the message */
$bound = '------------'.substr(strtoupper(md5(uniqid(rand()))), 0, 25);
$pgpmime .= "Content-Type: multipart/alternative;\r\n boundary=\"$bound\"\r\n\r\n";
$pgpmime .= "This is a multi-part message in MIME format.\r\n";
$pgpmime .= "--$bound\r\n";
$pgpmime .= "Content-Type: text/plain; charset=utf-8\r\n";
$pgpmime .= "Content-Transfer-Encoding: quoted-printable\r\n\r\n";
$pgpmime .= $clearText."\r\n\r\n";
$pgpmime .= "--$bound\r\n";
$pgpmime .= "Content-Type: text/html; charset=\"utf-8\"\r\n";
$pgpmime .= "Content-Transfer-Encoding: quoted-printable\r\n\r\n";
$pgpmime .= $clearContent."\r\n";
$pgpmime .= "--$bound--\r\n";
$content = GPG::cryptTxt($pgpkey, $pgpmime);
/* Make the email's headers */
$headers = '';
$headers = "From: $from\r\n";
$headers .= "Reply-to: ".$config['mailFrom']."\r\n";
$headers .= "X-Sender: WeSunSolve v2.0\r\n";
$headers .= "Message-ID: <".time()."@".$_SERVER['SERVER_NAME'].">\r\n";
$headers .= "Date: " . date("r") . "\r\n";
$bound = '------------enig'.substr(strtoupper(md5(uniqid(rand()))), 0, 25);
$headers .= "MIME-Version: 1.0\r\n";
$headers .= "Content-Type: multipart/encrypted;\r\n";
$headers .= " protocol=\"application/pgp-encrypted\";\r\n";
$headers .= " boundary=\"$bound\"\r\n\r\n";
/* And the cleartext body which encapsulate PGP message */
$mime = '';
$mime .= "This is an OpenPGP/MIME encrypted message (RFC 2440 and 3156)\r\n";
$mime .= "--$bound\r\n";
$mime .= "Content-Type: application/pgp-encrypted\r\n";
$mime .= "Content-Description: PGP/MIME version identification\r\n\r\n";
$mime .= "Version: 1\r\n\r\n";
$mime .= "--$bound\r\n";
$mime .= "Content-Type: application/octet-stream; name=\"encrypted.asc\"\r\n";
$mime .= "Content-Description: OpenPGP encrypted message\r\n";
$mime .= "Content-Disposition: inline; filename=\"encrypted.asc\"\r\n\r\n";
$mime .= $content."\r\n";
$mime .= "--$bound--";
mail($dest, $subject, $mime, $headers);

WeSunSolve: Site News April

New Features

  • Added wiki to hold the documentation;
  • Added the monitoring of multiple IPS Repositories;
  • User list now allows the user to load multiple patches at once;
  • Added patch timeline
  • Added CVE list affecting Solaris packages
  • Added patch link to CVE when issue is fixed
  • Users can now download a ZIP containing all README of user patch list
  • SSL Signed certificate added to wesunsolve.net https domain
  • User login goes over SSL by default
  • Added support for SRV4 Packages link to patches
  • Modified the structure of patch level to link SRV4 packages
  • Added user setting for API access
  • Added function to API to allow server registration and adding a patch level
  • Added patch/security report based on patch level and PCA execution
  • Added mail report for patch/security based on a patch level and patchdiag.xref automatic selection
  • We added a logo to our Wiki! (thanks to Dagobert Michelsen for the logo ;-)

Full listing of changes made can be found here

Patch level report (Using PCA)

PCA has been integrated into WeSunSolve so you can generate patch report on any server registered into your account where at least one patch level is defined.
The report which is created by WeSunSolve is based on the information you are entering when adding a server's patch level: showrev-p.out and pkginfo-l.out.
Theses two files are generated while running the Explorer or simply gathered by hand with the two corresponding commands. (respectively: /usr/bin/showrev -p and /usr/bin/pkginfo -l).

You can see there a full example of such generated report.
To generate a report like this, you must Add a server and an associated patch level, you can achieve this by following steps pointed in the documentation.

Please, give us feedback if you feel something is missing inside this report!

Mail reports

You can also get the previous report being sent to you by mail regularly, everything can be configured to fit your needs... You can:

  • Choose the server and the patch level on which the report will be generated;
  • Choose the interval between two reports being sent to you: every day, every week, every month ?
  • You can decide which patchdiag.xref delay you want to have, this is the best if you always want to have a delay between what's out and what you will actually install.

This way, you can get a report of what patches are to be installed on your server based on an up-to-date baseline every day...

To create a report, simply follow the steps at our documentation.

API Access

As of now, you can enable the API access inside your panel and take advantage of the function we have recently implemented, like:

  • Add a server easily;
  • Upload a patch level directly from command line;
We plan to add more feature to the API very soon...

Least known features: Window size

If you are browsing WeSunSolve regularly, you can greatly enhance your browsing by fitting the size of the website to your resolution.
We've implemented three size of screen:

  • 960px
  • 1200px
  • 1600px
By default, the website is rendering in 960px, which is fine to cope with most of our visitors but certainly not the best one if you have a 22" screen ;-)
See our documentation to know how to change your settings.

Like it? Spread it!

Please, if you do like WeSunSolve, spread it over your fellow sysadmin! Write a blog post 'bout it and send it over to get a backlink :-)

You found a cool way of doing something with WeSunSolve that spared you hours of work? Please, tell us how! Don't hesitate to write a Howto on our wiki...

Finally, if you want to thank me personally, you can simply connect through LinkedIN and let a little recommendation on the WeSunSolve job...

One Liner: Ifstat

Recently I tried ifstat on a freshly updated linux box with a 3.2 kernel, it was reporting nothing (0kb) although the network was heavily used.

A quick one-liner confirmed this and I decided to keep it here for later use:

export IFACE=eth0 ; while(:); do cat /proc/net/dev;sleep 1;done|nawk -W interactive -v iface=${IFACE} 'BEGIN{ cnt = 0 } $1 ~ iface { if (cnt) { print iface ":",(($2 - cnt)/1024),"Kbytes/sec"; } cnt=$2; }'