A Sneak Peek into the Protocol Behind Perforce
By Ixia Blog Team | Perforce[1] is a central server-based proprietary versioning control system. The Perforce server manages a central database and a master file repository that provides version tracking.
In order to communicate with the server, Perforce clients use a proprietary RPC and streaming protocol that runs on top of TCP/IP. The client application itself is thin in functionality, having most of its actions directed from the server (e.g., almost all messages printed to standard output come from the server via the RPC protocol).
RPC Protocol Structure
The general structure of the RPC protocol behind Perforce is presented in Figure 1, below. Each RPC call is composed of:
- A call header containing:
- one byte representing the XOR of the subsequent two length bytes
- the total length (in bytes) of the call parameters following the header, represented as a two byte little endian integer
- two unknown bytes, set to 0
- One or more call parameters that encode the RPC method arguments, along with the method name. (The method name is always encoded the last parameter with the name, func, and the value the actual method name.)
Fig. 1. Perforce RPC protocol structure
Each parameter is encoded as a <name, value> pair with the following fields:
- A variable length string representing the parameter name
- A null byte indicating the end of the name string
- A four byte little endian integer representing the length of the parameter value
- A variable length byte string representing the parameter value
- A null byte indicating the end of the parameter
An example of a decoded RPC call can be found in Fig. 2.
Fig. 2. Example of a decoded Perforce RPC call with highlighted fields
Hexinator, with a custom grammar we’ve created for Perforce, has been used for the actual decoding of the protocol. You can download the grammar here [2].
As we can see in the decoded figure above, the RPC call has three parameters—fseq=176, himark=0 and func=flush2—and is equivalent to calling flush2(176, 0). The flush2 function is used to indicate that a response to the previous RPC calls the client has made is expected from the server side.
Perforce Operations
In the following sections, we will look at how different Perforce workflows (login, workspace synchronization, changelist creation & submission) operate from an RPC call perspective.
Ticket Based Authentication
Perforce uses a challenge-response ticket-based authentication mechanism, where the client first initiates the authentication process with the server, the server sends back a challenge payload and then the client sends back a challenge response (i.e, an MD5 hash of the client’s password and the challenge information). Finally the server, upon validating the challenge response, grants a session ticket or token to the clients that can be used in subsequent requests. The session ticket has a server-configured expiry time (typically, about eight hours). After the expiry time has passed, the session ticket is no longer valid and the client must re-authenticate with the server in order to obtain a new ticket. The RPC call ladder diagram for the authentication process is presented in Figure 3, below.
Fig. 3. RPC call ladder of the Perforce authentication process
The process begins with the client sending two RPC calls: (1) protocol, in order to negotiate Perforce-specific protocol parameters (e.g., send buffer and receive buffer) followed by (2) user-login, where the client provides a username, host, workspace identifier and OS version to the server.
The server responds back with a similar protocol RPC call, i.e., a client-Prompt call where the challenge payload is located (mangle=F1E32334A78F1FB013EFF6AB9F77CDF9) as well as a flush call indicating that it is now expecting the client to send the next RPC call.
The client then proceeds to send back the challenge response
(data=6FE06650DE744731D54B5C110B352CB60D7F9EA103B7B982AB9F101C4EA58C82) via the dm-Login RPC call, after which is a flush call indicating that the server is next in line to respond.
Finally, the server sends back the session token via the client-SetPassword RPC call (data=ABE164F14FF126C8B9EE165DF40DCF7A) and instructs the client to display a “user logged in” message via the client-Message RPC call.
In the end, the parties exchange release RPC calls, which instruct the other side that the current Perforce connection should be terminated.
File Synchronization
In the case of local workspace file synchronization, the client provides a sync path to the server for each file that is to be synced. Then the server issues several RPC calls to the client, which create and transmit the actual file contents to the local workspace.
Fig. 4. RPC call ladder for Perforce file synchronization
The process starts off with corresponding protocol RPC calls, after which the client provides a synchronization path via the user-sync RPC call. The server then requests the authorization token from the client via the client-Crypto RPC call, and the client provides the token via the crypto call.
Following authorization, the server calls client-Message to indicate the provided path is currently syncing. The server then calls client-OpenFile (which opens a file for writing in the local client workspace), followed by one or more client-WriteFile RPC calls, which transfer the file data in 4096 byte chunks.
After all data is transmitted, the server calls client-CloseFile to close the file and client-Ack to request an acknowledgement from the client that all of the file data has been received.
Finally, the client calls dm-TookFile, which confirms to the server that the file is now synced correctly to the client’s workspace.
Changelist Creation
When creating a new changelist, the Perforce client asks the server for a changelist specification template, the server provides the template and then the client updates the changelist specification (e.g., changelist description).
Fig. 5. RPC call ladder for Perforce changelist creation
To begin the process, the client issues a user-change RPC call indicating to the server that it wishes to create a new changelist. Then, after the normal protocol and client-Crypto exchanges, the server sends back the changelist specification template via the client-EditData RPC call. The client then sends back the updated changelist specification via the dm-UpdateChangeSpec RPC call. Finally, the server issues a message to the client, via the client-Message RPC call, indicating that the changelist has been created.
Changelist Submission
The changelist submission process begins with the client indicating to the Perforce server that it wishes to submit a specific changelist. The server then indicates to the client that it should start sending new or modified files from the changelist. The client then sends the required files in chunks to the server. Finally, the client confirms to the server that all changelist files have been transmitted and that the changelist can be committed.
Fig. 6. RPC call ladder for Perforce changelist submission
The client intent is captured in the user-submit RPC calls, which instructs the server to initiate the process of submitting the specified changelist. The server then issues several messages to the client, indicating the different stages in the submission process (e.g., submitting change, locking files, showing which files will be transferred). The server then proceeds with a client-SendFile RPC call, which instructs the client to send a specific file.
The client then issues a dm-LbrOpen RPC call to open the specified file for writing on the server and proceeds to transfer the file contents in chunks via lbr-WriteFile RPC calls. When all of the file data has been transferred, the client calls dm-SubmitFile. The server responds back with client-Ack calls, indicating that it is awaiting a dm-CommitSubmit confirmation call from the client after all files has been transferred in order to actually commit the changelist to the repository.
Finally, the client issues the dm-CommitSubmit RPC call and the server calls back client-Message to indicate the specified changelist has been committed.
Perforce Support In Ixia ATI
Ixia now offers Perforce protocol traffic emulation in two-arm mode via the Ixia Application Threat Intelligence Strikepack. We’ve also added a bunch of new Super Flows, demonstrating protocol capabilities, including: Add New File, Create Changelist, Submit Changelist, Ticket Based Authentication and Sync File.
Sync to our latest ATI Strikepack and start getting your hands dirty by testing your network devices with realistic Perforce traffic. Also, feel free to share any interesting insight you have about this proprietary protocol.
Fig. 7. Perforce Submit Changelist Super Flow
Leverage Subscription Service to Stay Ahead of Attacks
The Ixia BreakingPoint Application and Threat Intelligence (ATI) program provides bi-weekly updates of the latest application protocols and attacks for use with Ixia platforms.
Additional Resources:
View Ixia’s Full ATI Protocol List
References
[2] https://github.com/drcomaneci/hexinator_perforce_grammmar