LucidDbDocs
This is the main index page for documentation about the LucidDB open-source project, a DBMS optimized for business intelligence. LucidDB is part of The Eigenbase Project.
LucidDB User Documentation
Getting Started
- Getting Started - An Installation Guide
- Frequently Asked Questions
- LucidDB AdminUI - Web Based Server Management Tool
- A graphical browser-based frontend from DynamoBI specifically built for LucidDB
- A graphical SQL client for accessing many database platforms, including LucidDB
- A data modeling tool
User Guides
- User Defined Functions (UDF) Howto
- User Defined Functions (UDF) Java Howto
- User Defined Transformation (UDX) Java Howto
- Extract, Transform and Load (ETL) Tutorial
- Online Analytical Processing (OLAP)
- Concurrency Control
- Data Storage And Access
- Error Handling
- Index Access
- Join Optimization
- Join Implementation
- Slow Query Diagnosis: Diagnosing slow queries using EXPLAIN PLAN and other tools
- Subqueries
- Warehouse Labels
- Labels allow users to have read-only query access to a snapshot of the database at an earlier point in time. Snapshots are accessible regardless of concurrent activities on other connections, like ETL, preparing new data.
Administrator Guides
LucidDB is designed to require as little administration as possible, so we hope this section can remain short :)
- Beginner Best Practices
- Memory Management
- Buffer Pool Sizing
You should also increase the buffer pool sizing to increase the amount of memory that LucidDB uses to "run queries"
(the Java adjustments will only increase the catalog/parsing/optimizer memory settings but not the actual execution engine).
- Buffer Pool Sizing
- System Procedures
- Useful for: Backup/Restore, Generating DDL for tables and other objects, managing sessions.
- Useful information about tweaking: Memory usage, network settings, execution, and storage functionality.
- The DBA views defined by LucidDB provide a global picture of the contents of the system catalog (tables, columns, etc) and the volatile state of the system (sessions, cache utilization, etc).
- Performance Tuning
- Performance Counters
- Monitor Active monitoring
- Tracing Debug Logs in LucidDB
- Performing Backups and Restores
- How to upgrade to each new release.
- Running LucidDB as a Service / Daemon
- Running LucidDB as a Managed Java Bean
- Moving the LucidDB Database Catalog
- Scripting LucidDB using BASH shell scripts
Function and SQL Language Reference Guides
- AppLib - Description of the utility function library in LucidDB
- Supported Data Types
- SQL Standard Conformance
- SQL Language Reference
Connectors to External Data Sources (SQL/MED) (Management of External Data)
These connectors wrap up external datasources and allow them to present their data as foreign tables inside of LucidDB. Connectors are an adapters that generally allow normal SQL queries to be run against datasources that may not support the SQL language directly.
- Flat File Wrapper - Connector enabling read-only SQL accesses to text files
- JDBC Data Wrapper - Connector enabling presentation of external database tables in LucidDB
- Salesforce Data Wrapper - Connector for presenting Salesforce tables in LucidDB
- CouchDB connections:
- Blog: SQL access to couchdb views
- Blog: SQL access to CouchDB views : Easy Reporting
- Forum: Installing the CouchDB connector [1]
- Download the connector from [2]
Benchmark Studies
- TPCH Benchmarks
- SQBM
- OTP Benchmark using Bureau of Transportation Statistics, SSB Benchmark
- Benchmark Candidates - Published benchmarks and applications which could serve as good showcases for LucidDB's architecture
Commercial Support
Commercial support is no longer being offered. LucidDB is now fully out in the open and community supported.
Source on github https://github.com/LucidDB/luciddb
-- Monday, July 23, 2012
Quick, reliable service and support is available to you from DynamoBI, the commercial sponsor of LucidDB.
42nd Floor
601 Union Street
Two Union Square
Seattle, WA 98101 USA
Tel. +1 888 LucidDB (582-4332)
Time Zone: PDT
Business Hours: 8-6 PDT
[Contact us!]
Community Involvement
LucidDB is one of the best databases in the world because of the people, not just the technology.
- Community Support Forum
- Please visit, ask questions and help others
- [http://www.luciddb.org/forums/]
- Chat with us online
- Many LucidDB users hang out on an Internet Relay Chat channel (IRC) called ##luciddb, using Freenode's [3] servers.
- We would love to answer your questions and help you get up and going using LucidDB.
- When you connect to chat.us.freenode.net or other freenode server near your country, be sure to use the command /join ##luciddb to actually join the chat channel for LucidDB.
Software Name Platform Download Location qwebirc Web-based client http://webchat.freenode.net/ ChatZilla Firefox Add-on https://addons.mozilla.org/en-US/firefox/addon/chatzilla/ mIRC MS Windows http://www.mirc.com/ X-Chat MS Windows http://xchat.org/ X-Chat Linux http://xchat.org/ X-Chat Aqua Mac OS http://sourceforge.net/projects/xchataqua/ Colloquy Apple iPad & iPhone http://colloquy.info/ Android - Internet Relay Chat Google Android https://market.android.com/details?id=com.countercultured.irc&hl=en
- Bug and Feature Tracking
- Make a feature request or report a problem you have experienced.
- [http://jira.eigenbase.org/secure/Dashboard.jspa]
Presentations / Blogs
- Demonstrates loading data, defining an OLAP cube, and browsing the cube with Pentaho, Jaspersoft, SpagoBI and Saiku which all use Mondrian to drive their OLAP functionality.
- Webex Presentation and PDF of slides presenting benefits and features of LucidDB 0.9.3 in conjunction with Pentaho's BI Suite.
- DynamoBI (Commercial Sponsor of LucidDB) - CEO Nick Goodman - shares vision about how LucidDB forms an information hub from which many different data sources can be presented and queried through normal SQL.
Tutorial Videos and Screencasts
- DynamoBI - Getting started online videos.
- Installing and Loading LucidDB backed sample OLAP
- Video from this blog: Taking LucidDB for an OLAP Test-Drive - By Slawomir Chodnicki, on March 12th, 2011
- Installing and running LucidDB as a daemon in Linux using a Java Service Wrapper
- This is an approach that may also work well on Windows. There is another approach, which is shorter and easier using the BASH shell under Linux. Documentation for both approaches can be found at LucidDbAsDaemonService.
- Moving tables from one database to LucidDB
- Using Pentaho Data Integration to generate SQL for moving tables from an external database to LucidDB
- This link is actually to a document about upgrading version to version of LucidDB, but because a PDI transformation file is used to generate SQL for the transition, the process can be generalized as moving from an external database to LucidDB with only minor modifications to the transformation.
Experimental
- Pentaho Data Integration (PDI) - Bulk Loading Plugins
- Streaming Loader Step - Bulk load data the easy way. [RECOMMENDED]
- Bulk Load Step - [DEPRECATED after LucidDB 0.9.3]
- Streaming Loader : A good example for building bulk loaders from non-Java tools. [INFORMATIONAL]
- Horizontal Partitioning
- LucidDB does not currently support DDL for automatic horizontal partitioning of tables. Although LucidDB's column-store architecture implements automatic vertical partitioning, horizontal partitioning is still a useful technique for scaling up to very large data volumes, so this page describes some manual approaches which can be used for very simple schemas.
- The main client API for LucidDB is JDBC. This page discusses some options for overcoming the obstacle of developing an application in a language other than Java.
- Description of how to make use of some of the diff-based testing frameworks built into LucidDB for the purpose of test-driven ETL script development.
- Walk-through document showing how to combine LucidDB with Apache Hadoop's Hive.
Frequently Asked Questions
LucidDB Developer Documentation
- Building LucidDB from source code
- Building LucidDB from Perforce (version control software, VCS)
- Main Developer Collaboration Section
- Design Meetings - Feature Ideas and Roadmaps
LucidDB Design Documentation
Trademark Notice
LucidDB and the LucidDB logo are trademarks of DynamoBI Corporation.