The Geomap tool is a web app that renders your network topology on a geographical map, provided that you have seeded your room data with geographical coordinates.
The server-side code is in the
while the client-side code is in the directory
URLs and parameters¶
There are two types of resources in Geomap:
The web page showing the map (
Geographical network data in GeoJSON or KML format (
[variant] represents a variant name defined in the configuration
file. The URL
/geomap redirects to
[v] is the
first variant the user has access to.
Query string parameters¶
Position for center of map
Zoom level for map (0-18)
Description of which layers to show. For each layer in the map, one of the characters ‘B’ (base layer, displayed), ‘0’ (base layer, not displayed), ‘T’ (non-base layer, displayed), ‘F’ (non-base layer, not displayed).
Selected time interval for load data. Interval size (index, 1-5), dash, start time (
YYYYMMDDhhmm`). The interval sizes are: 1: month; 2: week; 3: day; 4: hour; 5: 10 minutes. (See ``media/js/geomap/TimeInterval.js)
The arguments (
time) are intended
to be used together. They specify (more or less) the complete state of the
user interface, and are used by the «Link to this configuration» link, which
sets up these arguments to reflect the current state.
bbox argument is not intended to be used together with the other
arguments (the arguments
zoom, if present, override
bbox argument). The
bbox argument is meant as a way for other
applications to be able to create links to Geomap for showing a certain area.
The data resource accepts the following parameters in the query string:
Data format for result, either
Bounding box of map area.
Size (in pixels) of map as shown in user agent.
How close (in pixels) two nodes may be before they are collapsed to one.
Time interval for load data, in the form expected by rrdfetch for its
Almost all the server-side code is involved with generating the data resource. The web page showing the map requires almost no server-side processing.
Data flows in pipeline style through the modules
output_formats; each of which
has as its main purpose the transformation of data from one form to another.
Except for the data representations which constitute the interfaces from one
part of the pipeline to the next, these modules are mutually independent. The
data flow is controlled by the function
conf module reads and parses Geomap’s configuration file. The
utils module provides general utility functions/classes which are
used freely in the other modules.
db module collects data from the database and RRD files based
on the query string arguments. The result is two dictionaries, representing
netboxes and connections, respectively. Each netbox is represented as a
dictionary; each connection as two dictionaries (one for each end).
graph.build_graph() function creates a graph structure from the
db module creates, while
removes uninteresting things from such a graph. The simplification consists
removing objects which are outside the viewing area; and
reducing the level of detail by collapsing sets of objects which are close to each other to single objects.
The resulting simplified graph contains pointers to all the original data in the form of a tree in each node (since nodes are collapsed in two stages, see below) and a list in each edge.
For nodes, the collapsing is done in two steps: First, all the netboxes in a single room are combined to one node. Next, rooms that are sufficiently close to each other are combined to “places”. After the nodes are collapsed thus, any edges with the same two places as their endpoints are combined to one edge.
features module converts a graph to a set of “features”,
i.e. nodes and lines with geographical coordinates. Each feature has an
associated style (color and width/radius) and a specification of a popup
box for the feature.
output_formats module converts a list of features to a string in
GeoJSON or KML format (for KML output, some information is lost).
Tricks to avoid reading RRD files: Cache, pseudo-laziness¶
Load data is read from RRD files. Each netbox/connection has its own file (each connection actually has two), so we may end up reading very many files. To avoid much file reading, we do two things:
Use a data structure inspired by lazy evaluation to avoid reading files which are not needed.
Cache values read from RRD files.
For 1, we use the
utils.lazy_dict class. An instance of
this class acts like a dictionary, but may contain values which are
not computed before they are looked up. This way, the code may be
written almost as if all the files were read in the beginning (one
must be a little careful to avoid unintentionally causing all values
to be evaluated), while only those files which turn out to be needed
are actually read.
For 2, we use Django’s caching framework. See the section labeled
util.js is not shown in the diagram (all files implicitly depend
on it). This file contains general utility functions which are used in other
files as if they were part of the standard library.
Most of the files provide somewhat more general functionality than what is
strictly needed in Geomap, and are intended to be mostly independent of each
other. The file
geomap.js instantiates all needed things from the other
files and connects them together.
The entry point for the client-side code is the function
init, defined in
geomap.js. This function is called when the page is loaded, through the
ONLOAD attribute on the
Any file whose name starts with an uppercase character defines a data type (class) of the same name (and defines few or no other names at the top-level). For some of the files which depend on OpenLayers, the data type defined is an extension of an OpenLayers class. For other files, the data type definition consists of a constructor function and a prototype object.
Any other file simply contains a collection of functions, and introduces no new named data types.
This may cause the NAV side of things to break if the OpenLayers API changes in a non-compatible way. On the other hand, keeping it at a fixed version has proved to be problematic because we include code from OpenStreetMap, and this code apparently depends on the newest version of OpenLayers (shortly after OpenLayers 2.8 was released, using the OpenStreetMap code with OpenLayers 2.7 did not work).
There are two sets of online code documentation pages for OpenLayers: API documentation and documentation of everything. The first contains only the functions which are explicitly marked with “API” in the code. One should generally stick to the API documentation, since other functions are probably regarded as internal and likely to change. However, there seems to be some “API” labels lacking here and there, so sometimes it is useful to compare with the full documentation (or the source code).
API documentation for OpenLayers
Full documentation for OpenLayers
The reference to the file was found here: http://wiki.openstreetmap.org/wiki/OpenLayers_Simple_Example
We include the Proj4js library for coordinate transformations. We do not use this library directly, only through OpenLayers. (OpenLayers checks to see if Proj4js is available and uses it if it is).
The library is necessary to perform the conversions to/from UTM in
coordinates.js, which again is used by
PositionControl.js, which shows
the coordinates for a point the user clicked on the map.
Apart from the
ONLOAD attribute on
BODY, the HTML code (as it appears
call in an
set this up by modifying the DOM.
be present in the HTML code. The elements are generally adressed by
takes the id of the element as argument instead of having it hardcoded.
take a string used as common prefix for all ids as argument, and have
the remaining parts hardcoded. This strategy is used in
On the test system and test data used, generating the
/geomap/[variant]/data resource takes some time. In the best cases,
it takes one or a few seconds; in the worst, up to a minute.
The major cause (by far) of the long processing time is reading of RRD files. As discussed in the Server section above, we cache values from RRD files. This is the reason why the time varies a lot (the worst cases of time usage occur only with empty cache).
When moving or zooming the map, the new position will normally include much of the same data as the previous, so most of the needed RRD data will be in the cache, giving a “best case” processing time. When changing time interval or when first opening the map, on the other hand, the data is usually not in cache, giving a “worst case” processing time.
To improve the “best case” time, it is necessary to improve either the database queries or the Python code, or both. The very limited profiling which has been performed suggests that both the database queries and the subsequent processing of the results are responsible for their fair share of the total processing time. No “optimization” has been done on the Python code (although the programmer has tried to avoid extremely inefficient solutions), so there is probably some potential for performance improvement here. The database queries are large and hairy beasts (and will probably bite you if you appear threatening); whether (and if so, how) they can be made more efficient is hard to say.
To improve the “worst case”, the load data must simply be made available in a different form than RRD files so that it can be read faster.
Integration with Netmap¶
Some ideas for integration between Geomap and Netmap:
The popup boxes in the “normal” variant currently contain simple listings of all properties. This is convenient as an example of which properties are available and how to get at them, but probably far from ideal for actual use. Better defaults should be provided based on what users actually want to see.
Various small issues¶
If (when) the server, for some reason, fails in generating the data resource, the network information simply disappears from the map, with no error message given to the user. This is probably not ideal, although users may not be very interested in hearing that a “GargleException occured on line 42 of obscurities.py” either. For development, the Web Developer Tools in either Firefox or Chrome are very convenient – its console lists all the URLs requested by the script, so it is easy to follow the last one in order to see what the server said.
setTimeoutfunction to update the user interface regularly.
If some users are interested in always seeing the newest data, it could be useful to have a most recent data selection as an alternative to selecting a specific time interval. When this selection is activated, the data could be updated regularly even when the map is not moved (use
When zooming far out, the network data has a tendency to disappear completely. This is probably caused by the fact that longitudes wrap around, so when the width of the map area is close to a multiple of the width of the whole world map, the difference between the longitude at the left and right edge is approximately zero. This confuses the code which filters out things that are outside the viewing area. It should not be very difficult to come up with a hack to fix this.
utils.fix()function has a known error (conveniently, none of the actual calls to the function cause this error to occur) marked with a TODO comment. It should probably be fixed. (No, the function is, despite the name, able to fix itself. Not in that sense, at least).