Problem

My client has a subversion repository used to store all the company applications. I have a sub-directory in this repo storing the code for my web applications. It includes CakePHP based web applications that have many folders with the svn:ignore property set to avoid tmp and other folders that have data that is not required in the repo. I do not have access to perform an svnadmin dump.

Solution

After a few minutes of searching Google and contemplating a solution, I decided to just write some Bash scripts to solve this problem. This is the procedure that I came up with:

1. Copy the old working sub-directory into the new working sub-directory
2. Remove all .svn from the old working sub-directory in the new working sub-directory
3. Change to the old working sub-directory in order to retrieve relative paths
4. Get a list of directories that have svn:ignore properties and put them into a file
5. Iterate through the list of directories
6. Establish a filename for storing the svn:ignore property information
7. Run the svn propget svn:ignore on the old working directory and output to the filename generated in step 3
8. Run the svn propset svn:ignore -F [filename] on the new working directory
9. Perform basic clean-up of temp files
10. Change back to directory the user was using when the script was called
11. Exit

#!/bin/bash

HOME="/path/to/home"
SRC="/path/to/old/working/directory"
DST="/path/to/new/working/directory"

cp -r "${SRC}" "${DST}"
find "${DST}" -type d -name .svn -exec rm -rf {} \;

pushd "${SRC}"

for i in `find . -type d | grep -v .svn`; do svn proplist $i | grep ^Prop | sed "s/^[^']*'\([^']*\)':/\1/"; done > "${HOME}/tmp"

while read line; do
 FILENAME=`echo $line | sed 's#/#_#g'`
 svn propget svn:ignore "${SRC}/$line" > "${HOME}/${FILENAME}.svnignore"
 svn propset svn:ignore -F "${HOME}/${FILENAME}.svnignore" "${DST}/$line"
 rm "${HOME}/${FILENAME}.svnignore"
done < "${HOME}/tmp"
rm tmp

popd

exit 0

Warning

This script was run in a Cygwin environment and has not been tested in other Unix or Linux environments. It makes the the assumption that the properties that are found are ONLY svn:ignore properties. Any others could cause issues. I only had svn:ignore properties set. At first I ran the first half of the real work in this script manually to see the outcome one step at a time. I later compiled into this single script, but without fully testing it again as my repo was already set and I did not want to destroy it.

Share/Save

I have yet to run across anyone in the Google search result world that is running a CakePHP application that contains multiple sub-applications. I have been developing multiple web-based applications in CakePHP that all reside in a single virtual host. The “master” application controls access to these applications. Each application is a sub-directory of the master APP/webroot (actually contains symlinks pointing back to the applications’ APP/webroot and Apache +FollowSymLinks).

Problem

My Session->flash notices do not work when I call a redirect within these sub-applications. If I do NOT redirect, then the flash appears as expected. The moment I redirect, it’s gone.

Solution

I went through a lot of “solutions” according to people experiencing the similar problems. In doing so, I noticed that I had all the Session.cookie parameters configured the same in every APP/config/core.php file.

Configure::write('Session.cookie', 'CAKEPHP');

I changed this for one of my sub-applications to something unique. Voila! It worked. I have not gone into the details as to why, but it seems apparent to me that the Session.cookie must be unique across the “master” application and any sub-applications residing within the “master” app. This allows individual session cookies to be started for each application to use/access. The “master” app is probably wiping the slate clean behind the scenes.

Configure::write('Session.cookie', 'APP_NAME');

Share/Save

I recently ran into an issue where I found it much more efficient to write a custom query to gather the data I needed to perform some processing than it was to utilize CakePHP’s model/controller normal data accessing functionality.

At first, I wanted to make sure that all of my related tables gathered all of the data from each other. This created numerous objects filled with data that I may or may not use. It also resulted in the need to loop through every array multiple times to gather the final data required.

So, I decided to make a custom query  in the model that would handle joining all the tables into one array. I ran into trouble where fields were being grouped into their respective models as expected by CakePHP’s model class. This caused a headache to access the data properly again.

Original Custom Query

SELECT
	environments.id,
	environments.name,
	deploy_files.name,
	web_services.name,
	CONCAT(web_services.name,\'.\',users.name,\'=\',users.password) line
FROM
	deploy_files,
	environments,
	users,
	web_services,
	users_web_services,
	users_environments,
	web_services_deploy_files
WHERE
users.id = users_web_services.user_id
AND web_services.id = users_web_services.web_service_id
AND users.id = users_environments.user_id
AND environments.id = users_environments.environment_id
AND web_services.id = web_services_deploy_files.web_service_id
AND deploy_files.id = web_services_deploy_files.deploy_file_id
AND users.active = 1
AND web_services.active = 1
AND deploy_files.active = 1
AND environments.active = 1
ORDER BY
	environments.name,
	deploy_files.name,
	web_services.name,
	users.hidden DESC,
	users.name

Original Output

array (
  0 =>
  array (
    'environments' =>
    array (
      'id' => '1',
      'name' => 'Production',
    ),
    'deploy_files' =>
    array (
      'name' => 'test.login',
    ),
    'web_services' =>
    array (
      'name' => 'WebService',
    ),
    0 =>
    array (
      'line' => 'WebService.security=on',
    ),
  )
)

This output is usable, but I would have preferred to have everything simplified a bit and not grouped into sub associative arrays

Solution

I noticed the results of the CONCAT in the original query were in their own sub-array (0, zero). So, I took every column and put them in their own CONCAT and gave them an alias.

New Custom Query

SELECT
	CONCAT(environments.id) env_id,
	CONCAT(environments.name) env_name,
	CONCAT(deploy_files.name) file_name,
	CONCAT(web_services.name) ws_name,
	CONCAT(web_services.name,\'.\',users.name,\'=\',users.password) line
FROM
	deploy_files,
	environments,
	users,
	web_services,
	users_web_services,
	users_environments,
	web_services_deploy_files
WHERE
users.id = users_web_services.user_id
AND web_services.id = users_web_services.web_service_id
AND users.id = users_environments.user_id
AND environments.id = users_environments.environment_id
AND web_services.id = web_services_deploy_files.web_service_id
AND deploy_files.id = web_services_deploy_files.deploy_file_id
AND users.active = 1
AND web_services.active = 1
AND deploy_files.active = 1
AND environments.active = 1
ORDER BY
	environments.name,
	deploy_files.name,
	web_services.name,
	users.hidden DESC,
	users.name

New Output

array (
  0 =>
  array (
    0 =>
    array (
      'env_id' => '1',
      'env_name' => 'Production',
      'file_name' => 'test.login',
      'ws_name' => 'WebService',
      'line' => 'WebService.security=on',
    ),
  )
)

Now everything is inside the single 0 sub-array. The biggest problem I see with this is that I have concocted a custom query that breaks any CakePHP naming conventions. My controller and other data consuming components now rely on KNOWING these specific columns from this query. However, it is a large query that generates a lot more data than has been shown in these examples. Therefore, I think it is more efficient and memory-saving than the CakePHP natural methods or even a basic custom query like the original custom query above.

Share/Save

Share/Save

1. Navigate to SharePoint Central Administration
2. Click Operations
3. Click Diagnostic Logging
4. Scroll to the bottom of the page to the Trace Log section
5. Change Number of log files to 1
6. Change Number of minutes to use a log file to 5

My client’s server is back up and running again, and I should not have to worry about the Trace Log getting out of control again.

The downside is that my Trace Log only goes back to the last 5 minutes of work on the server. I am researching my options here. It may not be important enough though.

Share/Save

http://blogs.msdn.com/sharepoint/archive/2007/05/14/understanding-the-sharepoint-calendar-and-how-to-export-it-to-ical-format.aspx

The key point is the query used:

SPQuery query = new SPQuery();
query.ExpandRecurrence = true;
query.Query =
    "<Where>" +
        "<DateRangesOverlap>" +
            "<FieldRef Name='EventDate' />" +
            "<FieldRef Name='EndDate' />" +
            "<FieldRef Name='RecurrenceID' />" +
            "<Value Type='DateTime'><Month /></Value>" +
        "</DateRangesOverlap>" +
    "</Where>";
// Look forward from the beginning of the current month
query.CalendarDate = new DateTime(DateTime.Now.Year, DateTime.Now.Month, 1); 

Share/Save

Today’s topic: a simple interface for checking out who has group memberships and where they have them. I came up with the idea after seeing the constant organizational changes and the employee role changes associated with them. So let’s talk about the two parts. If you just want the code without the breakdown, scroll to the bottom of the full article.

User Lookup

There are three methods to this (in addition to a base of methods that I find helpful with my custom pages). First let’s take a look at my basic header.

<%@ Page language="C#"
MasterPageFile="~masterurl/default.master"
SmartNavigation="true"
Inherits="Microsoft.SharePoint.WebPartPages.WebPartPage,
Microsoft.SharePoint,
Version=12.0.0.0,
Culture=neutral,
PublicKeyToken=71e9bce111e9429c"
meta:progid="SharePoint.WebPartPage.Document" %>
<%@ Import Namespace="Microsoft.SharePoint" %>


It really is this simple. You only need the Microsoft.SharePoint library! Now I begin my <script> section that contains the OnLoad override and other methods needed.

protected SPSite site;
protected SPWeb web;

protected override void OnLoad(EventArgs e)
{
try
{
site = new SPSite("http://server/");
web = site.OpenWeb("webdev");
SPGroup grpMembers = web.SiteGroups["Website Development Members"];
if(grpMembers.ContainsCurrentUser)
{
mvViewManager.ActiveViewIndex = 1;
runPage();
}
}
catch (Exception err)
{
litError.Visible = true;
litError.Text = err.Message;
}
}

The first thing I do is just a simple check to make sure that the user is in a group that is allowed to perform these tasks. I also wrap the entire thing in a try/catch just to make my life easier for error catching and display it in a Literal that is in the content of the page (you’ll see later). I have been playing around with ViewManagers lately, and that controls what content is displayed on the page. The content of the page is as follows.

<asp:Content ContentPlaceHolderID="PlaceHolderPageTitle" runat="server">
Web Development Admin Tools - User Audit
</asp:Content>
<asp:Content ContentPlaceHolderID="PlaceHolderMain" runat="server">
<asp:MultiView ID="mvViewManager" runat="server" ActiveViewIndex="0">
<asp:View ID="vNoPerms" runat="server">
<h3>You do not have permissions to view this page!</h3>
</asp:View>
<asp:View ID="vMain" runat="server">
<asp:DropDownList ID="ddlUsers" runat="server" AutoPostBack="true" OnSelectedIndexChanged="findUser"/>
<br/>
<br/>
<asp:Literal ID="litOutput" runat="server"/>
</asp:View>
</asp:MultiView>
<asp:Literal ID="litError" runat="server" Visible="false"/>
</asp:Content>

The litOutput and litError Literals are used for displaying information. The ddlUsers DropDownList is used for displaying a list of all users. As mentioned before, there are two views inside the ViewManager mvViewManager. The first view (index 0) is the default view. If the group evaluation from OnLoad passes, the second view (index 1) is displayed. The rest of this section is pretty self explanatory. Onto the methods that manipulate the DropDownList and output Literal.

protected void runPage()
{
if(Request.HttpMethod != "POST")
{
populateUsers();
}
}

protected void populateUsers()
{
foreach(SPUser user in site.RootWeb.AllUsers)
{
ListItem li = new ListItem();
li.Text = user.Name;
li.Value = user.ID.ToString();
ddlUsers.Items.Add(li);
}
}

protected void findUser(Object sender, EventArgs e)
{
SPUser user = site.RootWeb.AllUsers.GetByID(Convert.ToInt32(ddlUsers.SelectedItem.Value));
litOutput.Text = “”" + user.Name + “” belongs to the following groups:<br/><br/>”;
litOutput.Text += userMembership(user, site.RootWeb, “”);
}

protected string userMembership(SPUser user, SPWeb web, string webPath)
{
StringBuilder retVal = new StringBuilder();
//retVal.AppendFormat(“{0}<br/>”, web.Name == “” ? “Root” : web.Name);
webPath += (webPath == “” ? “” : “/”) + web.Name;
foreach(SPGroup group in web.Groups)
{
try
{
group.Users.GetByID(user.ID);
retVal.AppendFormat(“<b>{0}</b> – <a href=”removeUserFromGroup.aspx?uid={1}&gid={2}&web={3}”>REMOVE</a><br/>”,
group.Name,
user.ID,
group.ID,
webPath);
}
catch { }
}
foreach(SPWeb subweb in web.Webs)
{
retVal.AppendFormat(“{0}”, userMembership(user, subweb, webPath));
}
return retVal.ToString();
}
</script>

The methods we have here are: runPage(), populateUsers(), findUser(Object sender, EventArgs e), and userMembership(SPUser user, SPWeb web, string webPath). I try to keep my methods in the order in which they are called. This makes my life easier when debugging, etc. The first method called is runPage(). It is just a simple starter that does a check to make sure that the page was not a POST. If it was a POST, then someone must have accessed the page incorrectly or someone chose a user from the DropDownList. This avoids duplicate population of the user DropDownList, and improper access of the page. That takes us to populateUsers(). This method uses SPWeb.AllUsers to find all the users in the system and add them to the DropDownList. It makes their name the Text property and their userID the Value property. Notice the DropDownList gets the AutoPostBack property set to True and the OnSelectedIndexChanged property set to the findUser() method. This writes some header information to the output Literal, then kicks off a call to the recursive method userMembership. Using the SPUser object of the user chosen, SPWeb object of the current web in which we’re looking, and a string of the path to the web in which we’re looking, userMembership will traverse all webs and sub-webs in the site collection looking for memberships to groups in those webs. It will return a string that provides the group name and a link to remove that person from the group.

Removing Users from a Group

This page is fairly simple. It takes three GET parameters to determine which SPWeb to use, the SPUser.ID, and SPGroup.ID. The same permissions logic is used to determine if the person should be viewing this page. Then, it also makes sure that the Request.HttpMethod is set to GET. The runPage() method then takes these parameters and removes the user from that group. The code is below the userAudit.aspx code provided.

userAudit.aspx

<%@ Page language="C#"
MasterPageFile="~masterurl/default.master"
SmartNavigation="true"
Inherits="Microsoft.SharePoint.WebPartPages.WebPartPage,
Microsoft.SharePoint,
Version=12.0.0.0,
Culture=neutral,
PublicKeyToken=71e9bce111e9429c"
meta:progid="SharePoint.WebPartPage.Document" %>
<%@ Import Namespace="Microsoft.SharePoint" %>
<script type="text/c#" runat="server">
protected SPSite site;
protected SPWeb web;
protected override void OnLoad(EventArgs e)
{
try
{
site = new SPSite("http://server/");
web = site.OpenWeb("webdev");
SPGroup grpMembers = web.SiteGroups["Website Development Members"];
if(grpMembers.ContainsCurrentUser)
{
mvViewManager.ActiveViewIndex = 1;
runPage();
}
}
catch (Exception err)
{
litError.Visible = true;
litError.Text = err.Message;
}
}

protected void runPage()
{
if(Request.HttpMethod != “POST”)
{
populateUsers();
}
}

protected void populateUsers()
{
foreach(SPUser user in site.RootWeb.AllUsers)
{
ListItem li = new ListItem();
li.Text = user.Name;
li.Value = user.ID.ToString();
ddlUsers.Items.Add(li);
}
}

protected void findUser(Object sender, EventArgs e)
{
SPUser user = site.RootWeb.AllUsers.GetByID(Convert.ToInt32(ddlUsers.SelectedItem.Value));
litOutput.Text = “”" + user.Name + “” belongs to the following groups:<br/><br/>”;
litOutput.Text += userMembership(user, site.RootWeb, “”);
}

protected string userMembership(SPUser user, SPWeb web, string webPath)
{
StringBuilder retVal = new StringBuilder();
//retVal.AppendFormat(“{0}<br/>”, web.Name == “” ? “Root” : web.Name);
webPath += (webPath == “” ? “” : “/”) + web.Name;
foreach(SPGroup group in web.Groups)
{
try
{
group.Users.GetByID(user.ID);
retVal.AppendFormat(“<b>{0}</b> – <a href=”removeUserFromGroup.aspx?uid={1}&gid={2}&web={3}”>REMOVE</a><br/>”,
group.Name,
user.ID,
group.ID,
webPath);
}
catch { }
}
foreach(SPWeb subweb in web.Webs)
{
retVal.AppendFormat(“{0}”, userMembership(user, subweb, webPath));
}
return retVal.ToString();
}
</script>
<asp:Content ContentPlaceHolderID=”PlaceHolderPageTitle” runat=”server”>
Web Development Admin Tools – User Audit
</asp:Content>
<asp:Content ContentPlaceHolderID=”PlaceHolderMain” runat=”server”>
<asp:MultiView ID=”mvViewManager” runat=”server” ActiveViewIndex=”0″>
<asp:View ID=”vNoPerms” runat=”server”>
<h3>You do not have permissions to view this page!</h3>
</asp:View>
<asp:View ID=”vMain” runat=”server”>
<asp:DropDownList ID=”ddlUsers” runat=”server” AutoPostBack=”true” OnSelectedIndexChanged=”findUser”/>
<br/>
<br/>
<asp:Literal ID=”litOutput” runat=”server”/>
</asp:View>
</asp:MultiView>
<asp:Literal ID=”litError” runat=”server” Visible=”false”/>
</asp:Content>

removeUserFromGroup.aspx

<%@ Page language="C#"
MasterPageFile="~masterurl/default.master"
SmartNavigation="true"
Inherits="Microsoft.SharePoint.WebPartPages.WebPartPage,
Microsoft.SharePoint,
Version=12.0.0.0,
Culture=neutral,
PublicKeyToken=71e9bce111e9429c"
meta:progid="SharePoint.WebPartPage.Document" %>
<%@ Import Namespace="Microsoft.SharePoint" %>
<script type="text/c#" runat="server">
protected SPSite site;
protected SPWeb web;
protected override void OnLoad(EventArgs e)
{
try
{
site = new SPSite("http://server/");
web = site.OpenWeb("webdev");
SPGroup grpMembers = web.SiteGroups["Website Development Members"];
if(grpMembers.ContainsCurrentUser && Request.HttpMethod == "GET")
{
mvViewManager.ActiveViewIndex = 1;
runPage();
}
}
catch (Exception err)
{
litError.Visible = true;
litError.Text = err.Message;
}
}
protected void runPage()
{
int uid = Convert.ToInt32(Request.QueryString["uid"]);
int gid = Convert.ToInt32(Request.QueryString["gid"]);
litOutput.Text += uid + " " + gid;
SPWeb thisWeb = site.OpenWeb(Request.QueryString["web"]);
SPGroup group = thisWeb.Groups.GetByID(gid);
SPUser user = site.RootWeb.AllUsers.GetByID(uid);
thisWeb.AllowUnsafeUpdates = true;
thisWeb.Update();
group.Users.RemoveByID(uid);
thisWeb.AllowUnsafeUpdates = false;
thisWeb.Update();
litOutput.Text = """ + user.Name + "" REMOVED FROM: " + group.Name;
}
</script>
<asp:Content ContentPlaceHolderID="PlaceHolderPageTitle" runat="server">
Web Development Admin Tools - Remove User From Group
</asp:Content>
<asp:Content ContentPlaceHolderID="PlaceHolderMain" runat="server">
<asp:MultiView ID="mvViewManager" runat="server" ActiveViewIndex="0">
<asp:View ID="vNoPerms" runat="server">
<h3>You do not have permissions to view this page!</h3>
</asp:View>
<asp:View ID="vMain" runat="server">
<asp:Literal ID="litOutput" runat="server"/>
</asp:View>
</asp:MultiView>
<asp:Literal ID="litError" runat="server" Visible="false"/>
</asp:Content>

Share/Save

My current day job is developing SharePoint-based web applications. On Thursday, September 11 (coincidence), the SharePoint site became unresponsive. It was not storing any new data, but viewing existing data was still possible. This was important as I have end-users that rely on reports stored in Document Libraries. The daily reports would not save onto the website.

I’m sure that this article is SQL Server Administration 101 for most folks. For those diving into SharePoint with no DBA, this could be a lifesaver.

Table of Contents

Chapter 1: Transaction Log Deletion – what happens when I lost the WSS_Content transaction log
Chapter 2: The Recovery – the steps to recovering the corrupt MDF file
Chapter 3: Prevention – best practices for database and transaction log backup and truncation

Chapter 1: Transaction Log Deletion

After crawling around the servers a bit, I found out it was a disk space issue. Ends up that the Data drive for the SQL server that stores my SharePoint databases was full. I noticed a 136GB transaction log (LDF) file. I have not had formal SQL server training and the bulk of my experience is in the Linux world, so I was unsure of my next steps. Googling around I found people with similar issues and learned about sp_detach_db. I began to scramble as 8am was near and people would be coming into work. The detach was not working quickly enough. I shut down my SQL server processes and deleted the LDF file to start over on the transaction log. Little did I know that I just entered myself into a world of pain.

When the SQL server was brought back online, to my surprise, the WSS_Content database was toast. It would not allow access to it, and said that the database was corrupt. It was not shutdown cleanly… awesome news… lesson learned…

So, my first reaction is to call IT and ask for a restore of that file. More awesome news: “yea, the backups have been saying they were successful, but that file hasn’t been backed up…” Now I am worrying! I read further on Google results. I see forum replies saying things like, “well chances are if you lost your LDF file and the MDF won’t attach, the data is corrupt and you cannot get it back.”

I spend most of the day trying to find the right solution. Many of the answers on Google got me close, but did not work in the end. I find a piece of software called Stellar Phoenix SQL Database Recovery for $420. I think, well, I’ll try it and if it doesn’t work, I’ll get a refund. First problem I run into is the purchasing of the software. It takes 4 hours and 5 phone calls to get the software installed and the license working. They have a sales website that knows nothing about the software and their support team had me waiting on E-mails that never showed up. Again, lesson learned…

So, I finally have the software in hand. I rescue the MDF file, but to no avail. It transformed my 25GB MDF file into a 7MB MDF file. I apply for a refund (which is another story altogether). Soon, I learned to stop worrying and love SQL Server.

Chapter 2: The Recovery

At this point it is getting late in the evening and I am not getting anywhere. My brain is becoming mush, so I call it a night. The next morning I network around the office to learn of a colleague that is a SQL Server expert. She walks me through these steps and to my avail, the database comes back to life and SharePoint is happy. The icing on the cake is that I lost no data!

1. Make a copy of the MDF file to a remote location just in case.
2. Rename the MDF file on the SQL server.
3. Run: CREATE DATABASE WSS_Content (Double check that it puts the MDF and LDF files in the same location as they were before. Add the proper options if need be.)
4. Shutdown the SQL server.
5. Copy original database MDF file over the newly created MDF file.
6. Rename, Move or Delete the newly create LDF file to “hide” it from SQL.
7. Start SQL server.
8. Run: EXEC('ALTER DATABASE ''[WSS_Content]'' SET EMERGENCY');
9. Run: ALTER DATABASE [WSS_Content] SET SINGLE_USER;
10. Run: DBCC CHECKDB ([WSS_Content], REPAIR_ALLOW_DATA_LOSS) WITH NO_INFOMSGS;
11. Run: ALTER DATABASE [WSS_Content] SET MULTI_USER;

Chapter 3: Prevention

At this point, my transaction log would still grow to enormous proportions without the proper practices. My colleague and SQL Server expert showed me how to run backups and truncate the transaction logs daily.

• Run full database backups that overwrite themselves the next day.
  • Create a job for running full backups as so for each database that needs to be backed up:
    BACKUP DATABASE [WSS_Content] TO DISK='C:PATHTOBACKUP[WSS_Content].bak' WITH INIT
  • If the database needs to have the transaction log truncated, run this after the BACKUP DATABASE command:
    BACKUP LOG [WSS_Content] TO DISK='C:PATHTOBACKUP[WSS_Content]_LOG.bak'
  • Schedule the job to run once nightly such that it will end prior to any tape backups.
• Optional, run transaction log backups every 4 hours to ensure that data from 4 hours ago can be recovered in case of a problem throughout the day.
  • For each database that needs to have the transaction log backed up and truncated run:
    BACKUP LOG [WSS_Content] TO DISK='C:PATHTOBACKUP[WSS_Content]_LOG.bak'
  • Schedule the job to run every 4 hours and stop prior to the full database backup above.

Share/Save