Tuesday, July 28, 2009
iTextSharp.dll
iTextSharp has a patch for CE framework here . I have just applied the patch to 4.0.1 version to get the dll which can be downloaded here
Tuesday, January 27, 2009
Monday, January 05, 2009
Smartphone Programing
Have been looking up recently for articles on programming smartphone devices. Stopped upon http://weblogs.asp.net/nleghari/articles/smartphone.aspx and looks like a great first point to anyone whose wants to start programming smartphones. A few of the links in the original article are out of date and below is a reproduction of the article with dead links replaced.
Best of Articles :: Microsoft Smartphone Programming
[This (sort of) book is a collection of different articles combined under related chapters]
Chapter 1: Introduction to the .NET Compact Framework
Fundamentals of Microsoft .NET Compact Framework Development for the Microsoft .NET Framework Developer
<http://msdn.microsoft.com/en-us/library/aa446549.aspx>
What's New in Smartphone 2003
<http://msdn.microsoft.com/en-us/library/ms838097.aspx>
Develop for Windows Mobile 2003 for Smartphone Using the .NET Compact Framework
<http://msdn.microsoft.com/en-us/library/ms839369.aspx>
An Introduction to Microsoft SmartPhone, and Programming SmartPhone using C#
<http://www.codeproject.com/netcf/SmartphoneIntroCSharp.asp>
Design Considerations for Microsoft Smartphone Applications
<http://www.oreillynet.com/pub/a/wireless/2004/01/07/smartphone.html>
Chapter 2: The Smartphone User Interface
Write Apps for the Smartphone without Dumbing Down Your UI
<http://www.devx.com/wireless/Article/21291>
Creating a Multiple Form Application Framework for the Microsoft .NET Compact Framework
<http://msdn.microsoft.com/en-us/library/aa446546.aspx>
How to Create a Microsoft .NET Compact Framework-based Image Button
<http://msdn.microsoft.com/en-us/library/aa446518.aspx>
Crafting Smartphone User Interfaces Using .NET Compact Framework
<http://msdn.microsoft.com/en-us/library/aa446513.aspx>
Develop for the SmartPhone Using Techniques You Already Know
<http://www.devx.com/wireless/Article/17968>
Chapter 3: Smartphone Controls
Creating Controls by Hand in the .NET Compact Framework
<http://www.devx.com/getHelpOn/10MinuteSolution/17430>
.NET Compact Framework Sample: Hosting A Native Windows Control
Developing Custom Controls in C# with Smart Device Extensions
<http://msdn.microsoft.com/en-us/library/ms836776.aspx>
Chapter 4: Graphics
Creating a Microsoft .NET Compact Framework-based Animation Control
<http://msdn.microsoft.com/en-us/library/aa446483.aspx>
Chapter 5: Files and Directories
Working with files on Smartphone devices with the .NET Compact Framework
<http://msdn.microsoft.com/en-us/library/aa446567.aspx>
Chapter 6: Mobile Web Services
Consuming Web Services with the Microsoft .NET Compact Framework
<http://msdn.microsoft.com/en-us/library/aa446547.aspx>
Chapter 7: Working with Unmanaged Code
Accessing Phone APIs from the Microsoft .NET Compact Framework
<http://msdn.microsoft.com/en-us/library/aa446547.aspx>
Advanced P/Invoke on the Microsoft .NET Compact Framework
<http://msdn.microsoft.com/en-us/library/aa446529.aspx>
An Introduction to P/Invoke and Marshaling on the Microsoft .NET Compact Framework
<http://msdn.microsoft.com/en-us/library/aa446529.aspx>
Creating a P/Invoke Library
<http://msdn.microsoft.com/en-us/library/aa446550.aspx>
P/Invoking Serial APIs in the Compact Framework
<http://msdn.microsoft.com/en-us/library/aa446551.aspx>
Using dumpbin.exe as an Aid for Declaring P/Invokes
<http://msdn.microsoft.com/en-us/library/aa446532.aspx>
Chapter 8: Deploying Smartphone Application
Deployment Patterns for Microsoft .NET Compact Framework
<http://msdn.microsoft.com/en-us/library/aa446503.aspx>
Smartphone 2003 Application Deployment Demystified
http://msdn.microsoft.com/library/en-us/dnsmtphn2k3htmlsp_2003_app_deploy_demyst.asp?frame=true>
Chapter 9: Interoperability
Connecting Microsoft Mobile Devices to Java Infrastructures
<http://dotnet.sys-con.com/node/47342>
Architecting Disconnected Mobile Applications Using a Service Oriented Architecture
<http://msdn.microsoft.com/en-us/library/aa446503.aspx>
Chapter 10: Game Programming with Smartphone
Games Programming with Cheese: Part One
<http://msdn.microsoft.com/en-us/library/aa446511.aspx>
Games Programming with Cheese: Part Two
<http://msdn.microsoft.com/en-us/library/aa446552.aspx>
Games Programming with Cheese: Part Three
<http://msdn.microsoft.com/en-us/library/aa446553.aspx>
Games Programming with Cheese: Part Four
<http://msdn.microsoft.com/en-us/library/aa446554.aspx>
Chapter 11: Advanced Topics
Security:
A Practical Guide to the Smartphone Application Security and Code Signing Model for Developers
<http://msdn.microsoft.com/en-us/library/ms839377.aspx>
Sending and Receiving SMS Messages:
Receiving SMS Messages Inside a Managed Application
<http://msdn.microsoft.com/en-us/library/aa446561.aspx>
Sending SMSs from your Microsoft .NET Compact Framework-based Applications
<http://msdn.microsoft.com/en-us/library/aa446545.aspx>
Debugging:
Microsoft .NET Compact Framework Debugging
<http://msdn.microsoft.com/en-us/library/aa446495.aspx>
Performance:
An Overview of the .Net Compact Framework Garbage Collector
<http://weblogs.asp.net/stevenpr/archive/2004/07/26/197254.aspx>
Microsoft .NET Compact Framework Multi-threading Tips
<http://msdn.microsoft.com/en-us/library/aa446540.aspx>
Microsoft .NET Compact Framework Background Processing Techniques
<http://msdn.microsoft.com/en-us/library/aa446488.aspx>
Improving Microsoft .NET Compact Framework-based Application Form Load Performance
<http://msdn.microsoft.com/en-us/library/aa446535.aspx>
Developing Well Performing .NET Compact Framework Applications
<http://msdn.microsoft.com/en-us/library/aa446542.aspx>
.Net Compact Framework Advanced Memory Management
<http://weblogs.asp.net/mikezintel/archive/2004/12/08/278153.aspx>
Miscellaneous:
Creating Self-Updating Applications With the .NET Compact Framework
<http://msdn.microsoft.com/en-us/library/aa446487.aspx>
Improving .NET Compact Framework HTTP Communications using HttpWebRequest and Custom ASP.NET Providers
<http://msdn.microsoft.com/en-us/library/aa446517.aspx>
Connecting a Smartphone 2003 Application to a Remoting Infrastructure
<http://www.devx.com/dotnet/article/22460>
.NET Compact Framework Mobile Web Server Architecture
<http://msdn.microsoft.com/en-us/library/aa446537.aspx>
Chapter 12: What's Next
What's New in the .NET Compact Framework 2.0
<http://msdn2.microsoft.com/library/ws1c3xeh.aspx>
Other Resources (Links gathered from comments)
Pragmatic Smartphone Application Hints and Tricks:
http://homepages.inspire.net.nz/~gambit/Article/
Programming Smartphone 2002/2003/SE with Embedded Visual Basic (eVB):
http://www.omnisoft.com/articles/spevb/default.asp
Smartphone "Tip-o-the-Day"
http://spaces.msn.com/members/lesgainous/
Thursday, December 11, 2008
Scribles from OSDI
A few paper where I took notes - mostly first day papers.
dryadlinq:
how to write distributed data-parallel programs from ds.
pragramer writes program on a single machine and the program takes care of distributing code and running it
LINQ : Microsoft language INtegrate Query
LINQ is an interface - there are many excution engines - The prgram is sritten in Standard common interface.
Dryad: takes care of runnning the jobs on multiple machines - uses Files/TCP/FIFo for communication.
Very similar to map reduce.
The input documents could be located anywhere - SQL, disk, mem, ..
Generate a distributed execution plan - similar to optimization in programing languages. These optimizations may help reduce the network consumption, .. Runtime code generation is used to optimize on the fly.
map reduce program can be easily converted to dryad linq interface. 3 lines of code
call mapper
sort intermediate values
call reducer
Dryadlinq is to be realeased for academic commujnity.
The main goal is completely opposite of virtual machines idea - Combine multiple computers and give a view of a single logical machine.
q. Ramki:
1. What are the syntax and symantic of this language?
There is no parallel expressions. You write the code in relational algebra
2. What are limitations - ex: recursion is hard to capture.
q. Google: Performace and intuition: Structuring programs is important since the same program could run in a second and could run in a huge amount of time:
We are still learning about it.
Everest: Scale down peak loads through I/O off-loading
Dushyanth Narayanan, Austin Donnelly, Eno Thereska, Sameh Elnikety, and Antony Rowstron, Microsoft Research Cambridge, United Kingdom
problem: IO peak in servers: Peak - short unexpected and high
motivate: IO trace from exchange servers.
Insert everest client on disk that we are bothered about. Monitor disk usage on each disk and offshore load during spikes to free disks.
Offloads writes during peaks. When the peak comes down - reclaim all the writes back to this disk.
properties of peaks: Uncorrelated across disks. Have writes - few foreground reads
Challenges: Want any write anywhere. Read should always return the correct last write. State must be consistent - apps would crash otherwise.
Keep metadata to track offloads - cached in memory.
Reclaiming: everest client when sees the disk is free - asks the cluster who has offloaded data.
Evaluated using exchange server trace - particularly three peaks that were observed. Second set of evaluation was done using OLTP benchmark.
Offload works for small high peaks and not to improve 24x7 performance.
q. Is there a synchronization problem that requires data on server to become persistent?
Wait for data to get persistent.
q. Can you configure what to offload or what to not? Yes that can be done at different levels.
q. How much correlations happens in volumes? Havent done it yet.
q. if you have JFS, do you do anything special for that? No
q. Would be using more write caches help? They would have to be very big - It would be the same - but have to have high cache
q. Underlying system has hardware raid? Does it introduce more failures levels? There are no more special dependencies introduced.
Improving MapReduce Performance in Heterogeneous Environments
Matei Zaharia, Andy Konwinski, Anthony D. Joseph, Randy Katz, and Ion Stoica, University of California, Berkeley
q. Why have global parameters - rather than looking at local scenario?
q. What do you do when you have a small cluster? Launch all primary tasks - only relaunch in the last run.
q. How would different run times of each map/reduce job would effect LATE? You need to normalize with something that says this is a small task and is a large task.
Corey:
Many applications spend time in kernel. Bottlenecks in OS is due to DS shared in kernel. Continuous design change is required to increase concurrency. Applications dont need to share all the ds that existing interfaces share.
Main idea: Reduce contention.
Shares: control the kernel data used to resolve application references.
Cost of using FD table: expected to scale linearly with number of cores. Throughput drops from 1 core to two cores and stays constant after that.
Reason: Additional core takes 121 cycles instead of 3 cycles. lock serialyzes updates to the table.
Use shares - allows applications to control how cores share the kernel ds used to do lookups. Let the application decide the fd tables.
Benefits: Apps can control how cores share internal kernel ds.
They also implement address ranges for each app. They benchmarked performance using map reduce. Using address ranges, apps can avoid contention.
CuriOS:
Problem: Errors occuring in systems due to hardware / software faults. Ex/l bit flips, stuck at faults , bad usage of pointers.
Managing OS errors: Currently - let the errors happen and go on to fix them later.
The fundamental problem with restart recovery - state from previous instance is gone after a crash. Save per client state in the server address space and in case of a crash save this information.
Redline: First Class Support for Interactivity in Commodity Operating Systems
Network Imprescision: A new consistency metric from scalable monitoring:
How to monitor large networks
Goals for monitoring: Scalability, real time moitoring, acuracy despite failures.
Half of the reports can differ by atleast 30 % from the truth. How to safegaurd accuracy despite disruption. Instead of giving best effort results give reliable results.
Quantify the stability of the system.
Define lower and upper bounds on number of nodes which are good and number of nodes which are bad (double counted) resp.
Nall, Nreachable, Ndup matrixes exposes impact of disruptions.
Chopstix: lightweight, high resolution monitoring for trouble shooting production systems
Run with low overheads - collect rich information about system .. to generate system states later in time.
hard bugs - happens once - have no clue why it happens...
Good copstix gui that can be used to dwelve down into the problem.
Better than oprofile in terms of overhead.
q. scalability of the chopstix - how many metrics can we capture? We havent carried out the numbers but it is now under 1% of CPU utilization.
Automating Network Application Dependency Discovery: Experiences, Limitations, and New Solutions
Xu Chen, University of Michigan; Ming Zhang, Microsoft Research; Z. Morley Mao, University of Michigan; Paramvir Bahl, Microsoft Research
Managing enterprise netwroks is hard. Need to have dependency information which could help in troubleshooting networks.
eXpose and sherlocks do this work.
contributions: use delay distribution.
use passive sniffing and only parse tcp/ip headers - minimize false dependencies.
System orion proposed - time delay between dependent services reflects typical processing and network delay.
Ignore some transient services. Only consider services that occur close in time.
Target 5 dominant applications in MS datacenters.
Have low FP when compared to sherlocks/expose.
FP comes from normal trends - like opening a web page from an email would realte mail server with proxy if the user clicks the page immediately.
dryadlinq:
how to write distributed data-parallel programs from ds.
pragramer writes program on a single machine and the program takes care of distributing code and running it
LINQ : Microsoft language INtegrate Query
LINQ is an interface - there are many excution engines - The prgram is sritten in Standard common interface.
Dryad: takes care of runnning the jobs on multiple machines - uses Files/TCP/FIFo for communication.
Very similar to map reduce.
The input documents could be located anywhere - SQL, disk, mem, ..
Generate a distributed execution plan - similar to optimization in programing languages. These optimizations may help reduce the network consumption, .. Runtime code generation is used to optimize on the fly.
map reduce program can be easily converted to dryad linq interface. 3 lines of code
call mapper
sort intermediate values
call reducer
Dryadlinq is to be realeased for academic commujnity.
The main goal is completely opposite of virtual machines idea - Combine multiple computers and give a view of a single logical machine.
q. Ramki:
1. What are the syntax and symantic of this language?
There is no parallel expressions. You write the code in relational algebra
2. What are limitations - ex: recursion is hard to capture.
q. Google: Performace and intuition: Structuring programs is important since the same program could run in a second and could run in a huge amount of time:
We are still learning about it.
Everest: Scale down peak loads through I/O off-loading
Dushyanth Narayanan, Austin Donnelly, Eno Thereska, Sameh Elnikety, and Antony Rowstron, Microsoft Research Cambridge, United Kingdom
problem: IO peak in servers: Peak - short unexpected and high
motivate: IO trace from exchange servers.
Insert everest client on disk that we are bothered about. Monitor disk usage on each disk and offshore load during spikes to free disks.
Offloads writes during peaks. When the peak comes down - reclaim all the writes back to this disk.
properties of peaks: Uncorrelated across disks. Have writes - few foreground reads
Challenges: Want any write anywhere. Read should always return the correct last write. State must be consistent - apps would crash otherwise.
Keep metadata to track offloads - cached in memory.
Reclaiming: everest client when sees the disk is free - asks the cluster who has offloaded data.
Evaluated using exchange server trace - particularly three peaks that were observed. Second set of evaluation was done using OLTP benchmark.
Offload works for small high peaks and not to improve 24x7 performance.
q. Is there a synchronization problem that requires data on server to become persistent?
Wait for data to get persistent.
q. Can you configure what to offload or what to not? Yes that can be done at different levels.
q. How much correlations happens in volumes? Havent done it yet.
q. if you have JFS, do you do anything special for that? No
q. Would be using more write caches help? They would have to be very big - It would be the same - but have to have high cache
q. Underlying system has hardware raid? Does it introduce more failures levels? There are no more special dependencies introduced.
Improving MapReduce Performance in Heterogeneous Environments
Matei Zaharia, Andy Konwinski, Anthony D. Joseph, Randy Katz, and Ion Stoica, University of California, Berkeley
q. Why have global parameters - rather than looking at local scenario?
q. What do you do when you have a small cluster? Launch all primary tasks - only relaunch in the last run.
q. How would different run times of each map/reduce job would effect LATE? You need to normalize with something that says this is a small task and is a large task.
Corey:
Many applications spend time in kernel. Bottlenecks in OS is due to DS shared in kernel. Continuous design change is required to increase concurrency. Applications dont need to share all the ds that existing interfaces share.
Main idea: Reduce contention.
Shares: control the kernel data used to resolve application references.
Cost of using FD table: expected to scale linearly with number of cores. Throughput drops from 1 core to two cores and stays constant after that.
Reason: Additional core takes 121 cycles instead of 3 cycles. lock serialyzes updates to the table.
Use shares - allows applications to control how cores share the kernel ds used to do lookups. Let the application decide the fd tables.
Benefits: Apps can control how cores share internal kernel ds.
They also implement address ranges for each app. They benchmarked performance using map reduce. Using address ranges, apps can avoid contention.
CuriOS:
Problem: Errors occuring in systems due to hardware / software faults. Ex/l bit flips, stuck at faults , bad usage of pointers.
Managing OS errors: Currently - let the errors happen and go on to fix them later.
The fundamental problem with restart recovery - state from previous instance is gone after a crash. Save per client state in the server address space and in case of a crash save this information.
Redline: First Class Support for Interactivity in Commodity Operating Systems
Network Imprescision: A new consistency metric from scalable monitoring:
How to monitor large networks
Goals for monitoring: Scalability, real time moitoring, acuracy despite failures.
Half of the reports can differ by atleast 30 % from the truth. How to safegaurd accuracy despite disruption. Instead of giving best effort results give reliable results.
Quantify the stability of the system.
Define lower and upper bounds on number of nodes which are good and number of nodes which are bad (double counted) resp.
Nall, Nreachable, Ndup matrixes exposes impact of disruptions.
Chopstix: lightweight, high resolution monitoring for trouble shooting production systems
Run with low overheads - collect rich information about system .. to generate system states later in time.
hard bugs - happens once - have no clue why it happens...
Good copstix gui that can be used to dwelve down into the problem.
Better than oprofile in terms of overhead.
q. scalability of the chopstix - how many metrics can we capture? We havent carried out the numbers but it is now under 1% of CPU utilization.
Automating Network Application Dependency Discovery: Experiences, Limitations, and New Solutions
Xu Chen, University of Michigan; Ming Zhang, Microsoft Research; Z. Morley Mao, University of Michigan; Paramvir Bahl, Microsoft Research
Managing enterprise netwroks is hard. Need to have dependency information which could help in troubleshooting networks.
eXpose and sherlocks do this work.
contributions: use delay distribution.
use passive sniffing and only parse tcp/ip headers - minimize false dependencies.
System orion proposed - time delay between dependent services reflects typical processing and network delay.
Ignore some transient services. Only consider services that occur close in time.
Target 5 dominant applications in MS datacenters.
Have low FP when compared to sherlocks/expose.
FP comes from normal trends - like opening a web page from an email would realte mail server with proxy if the user clicks the page immediately.
Saturday, November 08, 2008
Saturday, October 18, 2008
Compiling kernel > 2.6.13
If you see anything like this --
modprobe: FATAL: could not load /lib/modules.dep " NO such file or directory
mount:unknown filesystem type 'devfs'
FATAL : could not load /lib/modules/2.6.14/modules.dep:No such file or
directory
umount: devfs: not mounted
/scripts/ext3-add-journal.sh:27:arith:syntax error : "0X"
modprobe:FATAL : could not load /lib/modules/2.6.14/modules.dep:No such
file or directory
mount: unknown filesystem type 'devfs'
/sbin/init:426:arith:syntax error : "0X"
kernel panic - not syncing:Attempted to kill init!
You are using mkinitrd for making initrd file. Use mkinitramfs
It works ! Make sure to * cramfs
modprobe: FATAL: could not load /lib/modules.dep " NO such file or directory
mount:unknown filesystem type 'devfs'
FATAL : could not load /lib/modules/2.6.14/modules.dep:No such file or
directory
umount: devfs: not mounted
/scripts/ext3-add-journal.sh:27:arith:syntax error : "0X"
modprobe:FATAL : could not load /lib/modules/2.6.14/modules.dep:No such
file or directory
mount: unknown filesystem type 'devfs'
/sbin/init:426:arith:syntax error : "0X"
kernel panic - not syncing:Attempted to kill init!
You are using mkinitrd for making initrd file. Use mkinitramfs
It works ! Make sure to * cramfs
Monday, October 13, 2008
Installing Hadoop on Ubuntu
Have a cluster of 4 machines and am set to install hadoop on it.
Machines:
ds16: master
ds11: slave
ds04: slave
ds14: slave
Am loosely following sangmi's blog and the following
http://sangpall.blogspot.com/2008/09/installing-hadoop.html
http://wiki.apache.org/hadoop/GettingStartedWithHadoop
http://www.michael-noll.com/wiki/Running_Hadoop_On_Ubuntu_Linux_(Multi-Node_Cluster)
Step 1: chk if you have ssh, rsync and java on all machines
For java
sudo apt-get install sun-java5-jdk
Step 2:
Download hadoop and in conf/hadoop-env.sh
export JAVA_HOME=/usr/lib/jvm/java-1.5.0-sun
Step 3:
now you can just run your standalone operation as it is.
$ mkdir input
$ cp conf/*.xml input
$ bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'
$ cat output/*
[One problem faced was - when I did ssh to my machines, I get output
id: cannot find name for group ID 521
So I added a group with id 521 that solved the problem
sudo addgroup -gid 521 test
Also disabled stricthostchecking in /etc.ssh/ssh_config -- causing problem since ~ is mounted on nfs]
Till now we have run hadoop on single node. Lets move to a cluster now.
Cluster Operations:
Step 4:
Configure conf/hadoop-site.xml on the namenode as follows
Edit conf/masters and conf/slaves on ds16 as shown
[Used http://www.palfrader.org/code2html/code2html.html for converting code to html .. its nice]
[Make sure there are no leading spaces in the file -- gives a weird XML exception
[Fatal Error] hadoop-site.xml:2:6: The processing instruction target matching "[xX][mM][lL]" is not allowed.]
Step 5:
$mkdir /export/pathaka/hadoop/dfs
# This is the dfs file tree
$ mkdir /export/pathaka/hadoop/tmp
On master - ds16 run
$ bin/hadoop namenode -format
# This formats the dfs folder
#Do it only the first time and on master only
On ds16 - master, run
$ bin/start-dfs.sh
$ bin/start-mapred.sh
# master and jobtracker are the same machine
Stopping
$ bin/stop-mapred.sh
$ bin/stop-dfs.sh
Machines:
ds16: master
ds11: slave
ds04: slave
ds14: slave
Am loosely following sangmi's blog and the following
http://sangpall.blogspot.com/2008/09/installing-hadoop.html
http://wiki.apache.org/hadoop/GettingStartedWithHadoop
http://www.michael-noll.com/wiki/Running_Hadoop_On_Ubuntu_Linux_(Multi-Node_Cluster)
Step 1: chk if you have ssh, rsync and java on all machines
For java
sudo apt-get install sun-java5-jdk
Step 2:
Download hadoop and in conf/hadoop-env.sh
export JAVA_HOME=/usr/lib/jvm/java-1.5.0-sun
Step 3:
now you can just run your standalone operation as it is.
$ mkdir input
$ cp conf/*.xml input
$ bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'
$ cat output/*
[One problem faced was - when I did ssh to my machines, I get output
id: cannot find name for group ID 521
So I added a group with id 521 that solved the problem
sudo addgroup -gid 521 test
Also disabled stricthostchecking in /etc.ssh/ssh_config -- causing problem since ~ is mounted on nfs]
Till now we have run hadoop on single node. Lets move to a cluster now.
Cluster Operations:
Step 4:
Configure conf/hadoop-site.xml on the namenode as follows
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://ds16:54310</value>
</property>
<property>
<name>mapred.job.tracker</name>
<value>hdfs://ds16:54311</value>
</property>
<property>
<name>dfs.replication</name>
<value>8</value>
</property>
<property>
<name>dfs.name.dir</name>
<value>/export/pathaka/hadoop/dfs</value>
</property>
<property>
<name>mapred.child.java.opts</name>
<value>-Xmx512m</value>
</property>
</configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/export/pathaka/hadoop/tmp/</value>
<description>A base for other temporary directories.</description>
</property>
Edit conf/masters and conf/slaves on ds16 as shown
$ cat conf/masters
ds16
$ cat conf/slaves
ds16
ds04
ds11
ds14
[Used http://www.palfrader.org/code2html/code2html.html for converting code to html .. its nice]
[Make sure there are no leading spaces in the file -- gives a weird XML exception
[Fatal Error] hadoop-site.xml:2:6: The processing instruction target matching "[xX][mM][lL]" is not allowed.]
Step 5:
$mkdir /export/pathaka/hadoop/dfs
# This is the dfs file tree
$ mkdir /export/pathaka/hadoop/tmp
On master - ds16 run
$ bin/hadoop namenode -format
# This formats the dfs folder
#Do it only the first time and on master only
On ds16 - master, run
$ bin/start-dfs.sh
$ bin/start-mapred.sh
# master and jobtracker are the same machine
Stopping
$ bin/stop-mapred.sh
$ bin/stop-dfs.sh
Subscribe to:
Posts (Atom)