Problem scenario
You want to know if you have Apache Thrift correctly installed. What do you do?
Solution
Run this command: thrift -version
…
Continue reading “How Do You Check If Apache Thrift Is Installed?”
Question
What is Apache Parquet?
Answer
Apache Parquet is columnar data representation/manipulation tool for a Hadoop ecosystem. Data in a given column is largely uniform (e.g., a long string of characters, a single character, or an integer) in that it repeats a specific type and format of data as opposed to two cells in the same row (which may be very dissimilar types of data).
…
Problem scenario
You want to install Apache Parquet on the Hadoop namenode. What do you do?
Solution
Prerequisite
This assumes that you have installed Hadoop. For directions, see this posting.
Procedure
Run these commands:
sudo su –
apt-get -y install pip
pip install thriftpy
pip install snappy
exit
sudo apt-get -y install libsnappy-dev thrift-compiler
curl https://pypi.python.org/packages/74/b5/bc459aab0566fc3cf3397467922c37411ab6e3361bab9e0ca165e1089ce8/parquet-1.2.tar.gz#md5=05aacec0620ac63ecd7dd77bf7fb9fee >
…
Problem scenario
When trying to install Apache Parquet on Ubuntu, you get this error:
Running python-snappy-0.5.1/setup.py -q bdist_egg –dist-dir /tmp/easy_install-SIHL_T/python-snappy-0.5.1/egg-dist-tmp-O8UnkU
WARNING: ‘.’ not a valid package name; please use only .-separated package names in setup.py
package init file ‘__init__.py’ not found (or not a regular file)
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
snappymodule.cc:31:22: fatal error: snappy-c.h: No such file or directory
compilation terminated.
…
Problem scenario
You want to use Maven’s Apache Parquet plugin with Hadoop. How do you use these Apache technologies together?
Solution
1. Install HDFS. See this link if you are using Ubuntu. See this link if you are using a RedHat distribution of Linux. If you have more than one server and want a multi-node cluster of Hadoop, see this link for directions on how to deploy and configure it.
…
Continue reading “How Do You Configure Maven to Use an Apache Parquet Plugin?”
Problem scenario
You are not sure if Apache Parquet has been installed on your Linux server.
Solution
Run this command:
parquet –help
# This assumes you have logged out and logged back in after installing it. It assumes that the parquet exe is in the PATH environment variable too.
…
Continue reading “How Do You Know If Apache Parquet Is Installed?”
Problem scenario
You are trying to run Apache Parquet commands. But each command gives this error:
Traceback (most recent call last):
File “/usr/local/bin/parquet”, line 11, in <module>
load_entry_point(‘parquet==1.2’, ‘console_scripts’, ‘parquet’)()
File “/home/ubuntu/.local/lib/python2.7/site-packages/pkg_resources/__init__.py”, line 570, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File “/home/ubuntu/.local/lib/python2.7/site-packages/pkg_resources/__init__.py”, line 2750, in load_entry_point
raise ImportError(“Entry point %r not found” % ((group, name),))
ImportError: Entry point (‘console_scripts’,
…
Problem scenario
You want to install Apache Mesos and Apache Marathon to an Ubuntu 16.x Linux server in AWS. How do you do this?
Prerequisites
You need two Linux instances with relevant Security Group rules added to allow for connectivity between the two. One Ubuntu Linux server will be for the Mesos master and the other server will be for the Mesos slave. You also need to be able to use a web browser to the Mesos master server.
…
Problem Scenario: You want to use Ant to invoke your customized build.xml file. But when you run it, you get “
Target “build.xml” does not exist in the project …”
Solution: Call ant with no other variables. Do not use “ant build.xml”. Use “ant” by itself.
Miscellaneous: Most CentOS/RedHat repositories will have Ant available. As root, the command “yum install ant” should get it installed.
…
Continue reading “How to Use Ant (when You Receive an Error about build.xml Not Existing)”
“Documentation plays a great role in the maintenance of a project.” (Page 19, Learn Apache Ant, Tutorialspoint.com, 2014)
The open source movement is no exception. As much as we admire The Apache Software Foundation, we are disappointed that they don’t correct errors. We’ve submitted the first two errors below to them. However, mistakes remain. Here are some errors to help other people who may be confused:
#1 was corrected.
…
Continue reading “Apache Software Foundation Has Imperfect Websites”