Difference between revisions of "Solution to App Catalog Installation Limit"

From WebOS Internals
Jump to: navigation, search
Line 1: Line 1:
 
 
== Overview ==
 
== Overview ==
This solution involves creating a (hidden) directory in the /media/internal area, moving selected applications to the newly created directory, and then creating a symbolic link in the /var/usr/palm/applications directory pointing to the new location.  Version 0.3.0 and higher now also moves email and attachments to /media. This frees up the disk space from the relatively limited /var volume to the larger /media/internal. The 'mvapp' command (available on PreWare) will move the files and create the appropriate link.
+
This method involves creating a (hidden) directory in the /media/internal area, moving selected applications to the newly created directory, and then creating a symbolic link in the /var/usr/palm/applications directory pointing to the new location.  Version 0.3.0 and higher now also moves email and attachments to /media. This frees up the disk space from the relatively limited /var volume to the larger /media/internal. The 'mvapp' command (available on PreWare) will move the files and create the appropriate link.
  
 
== Solution (symbolic link method) ==
 
== Solution (symbolic link method) ==
  
 
'''Install Option 1''':<br>
 
'''Install Option 1''':<br>
 
+
Now on PreWare in the Linux Applications category. In PreWare Preferences, need to have 'show all package types' turned on.
Now on PreWare in the Linux Applications category, thanks to Rod Whitby.
 
  
 
- Install 'MvApp' from PreWare in the Linux Applications category.<br>
 
- Install 'MvApp' from PreWare in the Linux Applications category.<br>
Line 92: Line 90:
 
This will show largest app first, ask if you want to move/link and then moves on to the next largest app.  If you answer no to an app, it will skip to the next.  Answer 'q' to quit.  Is easiest to use this method directly on the phone using the Terminal app, available in Homebrew.
 
This will show largest app first, ask if you want to move/link and then moves on to the next largest app.  If you answer no to an app, it will skip to the next.  Answer 'q' to quit.  Is easiest to use this method directly on the phone using the Terminal app, available in Homebrew.
  
 +
'''To search for apps by partial app name'''
 +
 +
<code>mvapp search<br>
 +
(or shortcut)<br>
 +
mvapp s</code>
 +
 +
This is an easier way to link/unlink/clean apps from the Terminal app.  You only need to supply a partial app name and it will show any matches then ask what you'd like to do with it.
  
 
'''To revert the move and delete the link'''
 
'''To revert the move and delete the link'''
Line 212: Line 217:
 
- Some file attributes of linux fs are not possible on FAT fs (USB drive).  The 0.2.0 version (and higher) now deals with any apps that cannot copy attributes/permissions properly.  The script will not move apps that have special attributes or conditions that FAT fs can't deal with.  In general, this has been resolved.  No known issues with apps since version 0.2.0 or later.
 
- Some file attributes of linux fs are not possible on FAT fs (USB drive).  The 0.2.0 version (and higher) now deals with any apps that cannot copy attributes/permissions properly.  The script will not move apps that have special attributes or conditions that FAT fs can't deal with.  In general, this has been resolved.  No known issues with apps since version 0.2.0 or later.
  
- Some apps may not behave well if USB drive is mounted to computer, though I've tested several that behaved fine. Linux type background services probably would not work well so probably should not be moved, however I'm not aware of any service stored in the app home directory - they are usually setup somewhere else.  Apps that do IO to the home directory of the app while USB mounted may have issues when located on /media.  I'm not aware of any apps that write to their own home directory, so this may not be a risk in general, but is still a possibility.  If you run into issues with an app, see the Contingency Plan.
+
- Some apps may not behave well if USB drive is mounted to computer, though I've tested several that behaved fine. Linux type background services probably would not work well so probably should not be moved.  Apps that do IO to the home directory of the app while USB mounted may have issues when located on /media.  I'm not aware of any apps that write to their own home directory, so this may not be a risk in general, but is still a possibility.  If you run into issues with an app, see the Contingency Plan.
  
  

Revision as of 18:25, 8 November 2009

Overview

This method involves creating a (hidden) directory in the /media/internal area, moving selected applications to the newly created directory, and then creating a symbolic link in the /var/usr/palm/applications directory pointing to the new location. Version 0.3.0 and higher now also moves email and attachments to /media. This frees up the disk space from the relatively limited /var volume to the larger /media/internal. The 'mvapp' command (available on PreWare) will move the files and create the appropriate link.

Solution (symbolic link method)

Install Option 1:
Now on PreWare in the Linux Applications category. In PreWare Preferences, need to have 'show all package types' turned on.

- Install 'MvApp' from PreWare in the Linux Applications category.
- run 'mvapp' from the Terminal app (also available on PreWare) within the phone.

Install Option 2:
To install versions in development..

Enter Linux mode on Pre through a computer (paste the section below). Note that for test versions, the command is 'ma'.

mount -o remount,rw / 
cd /tmp
wget http://gitorious.org/webos-internals/mvapp/blobs/raw/master/mvapp
chmod 755 mvapp
mv mvapp /usr/local/bin/ma  #or name to whatever you'd like, using ma as shortcut
ma


Quick Start Guide
- Install 'mvapp' from PreWare
- Install Terminal app from PreWare (if not already installed)
- Open Terminal from launcher, then type...
- mvapp doctor
- mvapp bulkmv

Precautions

So far, no one has reported issues with apps or email due to linking. These are best practices to avoid issues.

- Version 0.3.0 and higher now can move email and attachments to media drive. It is unknown how the next webOS release will deal with the storage issues. It's possible the next release could break the email link to the media drive. While I'll likely have a new mvapp release out before next webOS release, losing email on the phone is a possibility and you may have to resetup email accounts. Moving email is not recommended if you rely on email for your job unless you don't mind possibly losing all email on the phone. Your email will still exist at the original source (gmail, yahoo, etc), so linking email is probably not very risky for most of us.

- If Palm releases a new webOS that resolves the overall issue, you may need to completely restore your apps back to /var before performing the upgrade - mvapp restoreall. For email - mvapp unlinkemail. If the webOS update is applied before you get a chance to restore, it's possible it could break this method, so moving apps that store important information to you is not recommended.

- This is a workaround script not approved by Palm. The usual disclaimers apply.

Commands

To run diagnostics on /var and get recommendations

mvapp doctor
(or shortcut)
mvapp d

This reports the usage of applications, email and attachments on /var. It also recommends actions based on the results.


To find the largest apps in /var/usr/palm/applications

mvapp list
(or shortcut)
mvapp ls

The first number is size of app in KB, largest shown last. IE....
8352 com.apnews.webos
8512 com.fandango.app.fandango
8672 com.palm.app.musicplayerremix
10304 com.shortcovers.palm.pre
10432 com.fusioncreativestudios.deadman
10656 com.ulocate.app.where


To move and link an app to /media

mvapp link domain.appname
(or shortcut)
mvapp ln domain.appname

Example: mvapp link com.ulocate.app.where

The app should now work in the new location thanks to the link. Test each app to make sure it works before doing another.

Continue moving apps until it reports that /var.../applications is about 30MB or less and /var has less than 90MB used.


To move and link many apps

mvapp bulkmv
(or shortcut)
mvapp bm

This will show largest app first, ask if you want to move/link and then moves on to the next largest app. If you answer no to an app, it will skip to the next. Answer 'q' to quit. Is easiest to use this method directly on the phone using the Terminal app, available in Homebrew.

To search for apps by partial app name

mvapp search
(or shortcut)
mvapp s

This is an easier way to link/unlink/clean apps from the Terminal app. You only need to supply a partial app name and it will show any matches then ask what you'd like to do with it.

To revert the move and delete the link

mvapp unlink domain.appname
(or shortcut)
mvapp ul domain.appname

The application will be moved back to the original directory. If you have issues with an app after unlinking, see the Contingency Plan section below.


To list apps that have been moved and linked...

mvapp listmoved
(or shortcut)
mvapp lm

This shows a list of apps that have been moved and linked to /media.


To restore all moved apps back to original location

mvapp restoreall
(or shortcut)
mvapp ra

This will restore ALL applications that have been moved/linked, back to the original location in /var. This may take several minutes to complete. It will show each application progress. Beware that if you have moved a very large number of apps over a period of time, they may not all fit on /var. You may need to remove some/many unused apps before restoring all.


To cleanup and remove directories and symlinks

mvapp clean domain.appname
(or shortcut)
mvapp c domain.appname

If you have any issues, first attempt to remove the app from Launcher, the App Catalog or Homebrew app installer. It's very important you remove from the phone first before using the clean command. Then issue the 'clean' command to remove the applications directories at both locations and also the symlink. You can then reinstall the app from the App Catalog or any Homebrew App installer.


To force tar restore

mvapp ftr domman.appname

This is a hidden function not shown in usage. It is for those who used an older version of mvapp and had tar backups. If you need to restore an app from tar, use this command. Or just remove the app from Launcher and reinstall if necessary.


To remove tar backups

mvapp rtb

This is a hidden function not shown in usage. Tar backups have been disabled by default since mvapp 0.2.4. Tar backup has been turned off because it wastes space on /media and has potential issues with restoring older versions of an app. Its purpose was to restore special file attributes not supported by FAT filesystem. The script no longer moves apps that FAT fs cannot support, so tar is no longer needed. If you no longer need old tar backups, use this command to remove the entire backup directory.

To move and link email and attachments

mvapp linkemail
(or shortcut)
mvapp le

This will move all of your email from /var to /media/internal/.data/emails. This will close the email application so if you are writing a new email, be sure to send the email before using this commend. Also see the Precautions section above.

To restore and unlink email and attachments

mvapp unlinkemail
(or shortcut)
mvapp ue

This will restore all of your email from /media back to /var. This will close the email application so if you are writing a new email, be sure to send the email before using this commend.

Contingency Plan

If you have problems with an application, follow these steps...
-- Close the application if open
-- mvapp unlink domain.appname
-- Try using the App

If the application still does now work correctly...
-- Remove the App using official methods (remove from Launcher or Homebrew installer)
-- mvapp clean domain.appname
-- Reinstall through the App Catalog or Homebrew installer app

If you have issues with email, follow these steps...
-- mvapp unlinkemail

Proposal to Dev Community

This is a formal proposal to the Dev Community suggesting that PreWare, WebOS Quick Install and other Pre installer apps provide an option to move any app in /var to the /media fs and create a link similar to the code above.

The Homebrew Community somewhat created part of the storage problem so needs to come up with their own solution. The symbolic link proponents propose that Homebrew apps be moved with a link to /media/internal by default and physically use /var only if needed (per conditions stated below). The developer would put a flag in the package (or some other method during submission) to state their app is able to run linked to /media or if it specifically needs to physically be on /var. Will propose additions to Packaging Standards to support /media links. The homebrew installer apps could then automatically do the move/link if the package is flagged for it.

Candidate apps for moving to /media
- apps that do not depend on file attributes not supported by FAT (such as sticky bit or internal links) - The script (post 0.2.0 now deals with this).
- apps that do not perform data operations to home app directory when device is USB mounted

Exceptions for maintaining apps on /var
- apps that depend on file attributes not supported by FAT - The script (post 0.2.0 now deals with this).
- apps that won't work well when device is USB mounted, such as performing data or DB operations in home app directory

(Please update with other known candidates/exceptions)

Benefits over other methods

Fair Dinkum method
Rod Whitby's FD method works up to a point but if /var usage is over 100MB, the FD method no longer works. The symlink method not only saves space on /var, it can reduce it significantly when many apps and email are moved to media drive.

Resizing /var
One challenge with resizing /var is that it will still have a fixed static limit - how do you decide how much to increase it? Many will still probably hit the limit or waste space if setting too high. There is also the warning from Palm that resizing var may interfere with future updates. The link method allows to dynamically use the /media partition, so there is no need to set a specific size dedicated to apps. If the USB drive is filled, users can decide if they use the space for media or apps on the fly.

AppPath in /etc/luna.conf
I proposed a while back adding an AppPath to luna.conf to include apps stored on /media. Some apps would not work because some apparently reference /var. IE, the vampire/mafia/quest series could not access graphics. The symbolic link fixed this because apps think they are in /var.

Link/mount all of /var/usr/palm/applications/ to /media/internal
If some apps rely on file attributes this won't work since file attributes are lost when moved to FAT fs. While this is rare, it probably isn't wise to force all /var apps to /media. Selectively moving apps one at a time is less risky. Update: Have found that "PDF View" app does not allow to be linked, so it appears that selectively moving apps is necessary.

Creating a loopback filesystem to a virtual file located on /media/internal
This has been worked on here and still has potential. Unfortunately it locks out USB mount and media sync altogether. If a workaround can be found with low risk, this may be the most ideal solution.

Suggestion to Palm... consider virtually mounting /media so that both the kernel and USB can use /media simultaneously. This would allow the loopback fs to work and would probably also allow phone calls while USB mounted. This could become the strategic solution.

Risks, Issues, Dependencies

- Some file attributes of linux fs are not possible on FAT fs (USB drive). The 0.2.0 version (and higher) now deals with any apps that cannot copy attributes/permissions properly. The script will not move apps that have special attributes or conditions that FAT fs can't deal with. In general, this has been resolved. No known issues with apps since version 0.2.0 or later.

- Some apps may not behave well if USB drive is mounted to computer, though I've tested several that behaved fine. Linux type background services probably would not work well so probably should not be moved. Apps that do IO to the home directory of the app while USB mounted may have issues when located on /media. I'm not aware of any apps that write to their own home directory, so this may not be a risk in general, but is still a possibility. If you run into issues with an app, see the Contingency Plan.


Confirmed Apps

Apps Confirmed Not to Work

com.palm.app.pdfviewer (script v0.2.0 and higher properly handles this, won't allow it to move)

Discuss

Discuss in the Discussion tab or PreCentral...

http://forums.precentral.net/web-os-development/205649-resolution-app-catalog-install-limit-proposal.html

Your Experiences

Please post your experiences, good or bad. I'd like to get any kinks worked out before attempting to turn this into a webOS app.

- How many apps did you move?
- Did you find an app that won't work linked?
- Did you move any back? (please test)
- How far down did you have to get /var.../applications down in MB before you could start adding apps from the App Catalog?

Source

The source has been moved to gitorious...

http://gitorious.org/webos-internals/mvapp

Credits

xorg - initially developed script and proposal.  maintainer of this page.
daventx - added unlink and clean functions, other suggestions<br>
nt4cats - found Fair Dinkum breaks mvapp, gave solution to use explicit path.
SirWill, hparsons, bclancy, navinag, dhcalva, chodaboy - testing, usable suggestions and feedback
emoney22 - pointed out hidden directory in /media hides images from Photo app
greg_roll - provided info to capture running apps
rwhitby - setup webosinternals repository for homebrew distro

And thanks to the many testers who gave feedback on the PreCentral thread.