Error: Timeout when trying to deploy

Automated deployment for web applications and databases

Moderators: Mike Upton, justin.caldicott, Sean.newham, csmith, chirayu, DavidSimner, david.conlin

Error: Timeout when trying to deploy

Postby lee5i3 » Mon Jul 08, 2013 6:03 pm

I have a project that has 14 steps (3 websites and rest are WCF web services)..

Deployment Manager is setup on a local box internally.. Agent is running on a Windows Azure VM

I did a health check and it passed with no problems.
It fails Stage 2 on \"Upload packages to target machines\"

Error says
The request channel timed out while waiting for a reply after 00:00:00. Increase the timeout value passed to the call to Request or increase the SendTimeout value on the Binding. The time allotted to this operation may have been a portion of a longer timeout.


Error Details:
2013-07-08 16:47:35 INFO Preparing to upload packages
2013-07-08 16:47:35 INFO Uploading packages...
2013-07-08 16:51:24 ERROR System.TimeoutException: The request channel timed out while waiting for a reply after 00:00:00. Increase the timeout value passed to the call to Request or increase the SendTimeout value on the Binding. The time allotted to this operation may have been a portion of a longer timeout. ---> System.TimeoutException: The request operation did not complete within the allotted timeout of 00:01:00. The time allotted to this operation may have been a portion of a longer timeout.
at System.ServiceModel.Channels.ReliableRequestSessionChannel.SyncRequest.WaitForReply(TimeSpan timeout)
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
--- End of inner exception stack trace ---

Server stack trace:
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at RedGate.Deploy.Shared.Contracts.IPackageService.UploadChunk(Byte[] data)
at CallSite.Target(Closure , CallSite , Object , Object )
at System.Dynamic.UpdateDelegates.UpdateAndExecuteVoid2[T0,T1](CallSite site, T0 arg0, T1 arg1)
at CallSite.Target(Closure , CallSite , Object , Object )
at ImpromptuInterface.Optimization.InvokeHelper.InvokeMemberAction(CallSite& callsite, Type binderType, LazyBinder binder, String_OR_InvokeMemberName name, Boolean staticContext, Type context, String[] argNames, Object target, Object[] args)
at ImpromptuInterface.Optimization.InvokeHelper.InvokeMemberActionCallSite(Object target, String_OR_InvokeMemberName name, Object[] args, String[] tArgNames, Type tContext, Boolean tStaticContext, CallSite& callSite)
at ImpromptuInterface.Impromptu.InvokeMemberAction(Object target, String_OR_InvokeMemberName name, Object[] args)
at ImpromptuInterface.Dynamic.ImpromptuForwarder.TryInvokeMember(InvokeMemberBinder binder, Object[] args, Object& result)
at RedGate.Deploy.Server.Proxies.WcfSecurityWrapper`1.TryInvokeMember(InvokeMemberBinder binder, Object[] args, Object& result)
at CallSite.Target(Closure , CallSite , Object , Byte[] )
at ActLike_IPackageService_2d858676b1eb41aa9c0a21844e982fb8.UploadChunk(Byte[] data)
at RedGate.Deploy.Server.Tasks.Deploy.PackagePusher.UploadFile(IPackageService service, String fullPath, ILog log)
at RedGate.Deploy.Server.Tasks.Deploy.PackagePusher.Push(DownloadedPackage package, MachineEndpoint destination, ILog log)
at RedGate.Deploy.Server.Tasks.Deploy.UploadPackageActivity.<Execute>b__0()
at System.Threading.Tasks.Task.InnerInvoke()
at System.Threading.Tasks.Task.Execute()

Exception rethrown at [1]:
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccess(Task task)
at RedGate.Deploy.Server.Tasks.ActivityRuntime.<ExecuteChild>d__0.MoveNext()
2013-07-08 16:51:44 ERROR System.TimeoutException: The operation did not complete within the allotted timeout of 00:00:00. The time allotted to this operation may have been a portion of a longer timeout.

Server stack trace:
at System.ServiceModel.Channels.InterruptibleWaitObject.Wait(TimeSpan timeout, Boolean throwTimeoutException)
at System.ServiceModel.Channels.ReliableOutputConnection.Close(TimeSpan timeout)
at System.ServiceModel.Channels.ReliableRequestSessionChannel.OnClose(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Close(TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.OnClose(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Close(TimeSpan timeout)
at System.ServiceModel.Dispatcher.TerminatingOperationBehavior.AfterReply(ProxyRpc& rpc)
at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at RedGate.Deploy.Shared.Contracts.IPackageService.FinalizeUpload()
at CallSite.Target(Closure , CallSite , Object )
at System.Dynamic.UpdateDelegates.UpdateAndExecuteVoid1[T0](CallSite site, T0 arg0)
at CallSite.Target(Closure , CallSite , Object )
at ImpromptuInterface.Optimization.InvokeHelper.InvokeMemberAction(CallSite& callsite, Type binderType, LazyBinder binder, String_OR_InvokeMemberName name, Boolean staticContext, Type context, String[] argNames, Object target, Object[] args)
at ImpromptuInterface.Optimization.InvokeHelper.InvokeMemberActionCallSite(Object target, String_OR_InvokeMemberName name, Object[] args, String[] tArgNames, Type tContext, Boolean tStaticContext, CallSite& callSite)
at ImpromptuInterface.Impromptu.InvokeMemberAction(Object target, String_OR_InvokeMemberName name, Object[] args)
at ImpromptuInterface.Dynamic.ImpromptuForwarder.TryInvokeMember(InvokeMemberBinder binder, Object[] args, Object& result)
at RedGate.Deploy.Server.Proxies.WcfSecurityWrapper`1.TryInvokeMember(InvokeMemberBinder binder, Object[] args, Object& result)
at CallSite.Target(Closure , CallSite , Object )
at ActLike_IPackageService_2d858676b1eb41aa9c0a21844e982fb8.FinalizeUpload()
at RedGate.Deploy.Server.Tasks.Deploy.PackagePusher.UploadFile(IPackageService service, String fullPath, ILog log)
at RedGate.Deploy.Server.Tasks.Deploy.PackagePusher.Push(DownloadedPackage package, MachineEndpoint destination, ILog log)
at RedGate.Deploy.Server.Tasks.Deploy.UploadPackageActivity.<Execute>b__0()
at System.Threading.Tasks.Task.InnerInvoke()
at System.Threading.Tasks.Task.Execute()

Exception rethrown at [1]:
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccess(Task task)
at RedGate.Deploy.Server.Tasks.ActivityRuntime.<ExecuteChild>d__0.MoveNext()
2013-07-08 16:51:49 ERROR System.TimeoutException: The request channel timed out while waiting for a reply after 00:00:00. Increase the timeout value passed to the call to Request or increase the SendTimeout value on the Binding. The time allotted to this operation may have been a portion of a longer timeout. ---> System.TimeoutException: The request operation did not complete within the allotted timeout of 00:01:00. The time allotted to this operation may have been a portion of a longer timeout.
at System.ServiceModel.Channels.ReliableRequestSessionChannel.SyncRequest.WaitForReply(TimeSpan timeout)
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
--- End of inner exception stack trace ---

Server stack trace:
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at RedGate.Deploy.Shared.Contracts.IPackageService.FinalizeUpload()
at CallSite.Target(Closure , CallSite , Object )
at System.Dynamic.UpdateDelegates.UpdateAndExecuteVoid1[T0](CallSite site, T0 arg0)
at CallSite.Target(Closure , CallSite , Object )
at ImpromptuInterface.Optimization.InvokeHelper.InvokeMemberAction(CallSite& callsite, Type binderType, LazyBinder binder, String_OR_InvokeMemberName name, Boolean staticContext, Type context, String[] argNames, Object target, Object[] args)
at ImpromptuInterface.Optimization.InvokeHelper.InvokeMemberActionCallSite(Object target, String_OR_InvokeMemberName name, Object[] args, String[] tArgNames, Type tContext, Boolean tStaticContext, CallSite& callSite)
at ImpromptuInterface.Impromptu.InvokeMemberAction(Object target, String_OR_InvokeMemberName name, Object[] args)
at ImpromptuInterface.Dynamic.ImpromptuForwarder.TryInvokeMember(InvokeMemberBinder binder, Object[] args, Object& result)
at RedGate.Deploy.Server.Proxies.WcfSecurityWrapper`1.TryInvokeMember(InvokeMemberBinder binder, Object[] args, Object& result)
at CallSite.Target(Closure , CallSite , Object )
at ActLike_IPackageService_2d858676b1eb41aa9c0a21844e982fb8.FinalizeUpload()
at RedGate.Deploy.Server.Tasks.Deploy.PackagePusher.UploadFile(IPackageService service, String fullPath, ILog log)
at RedGate.Deploy.Server.Tasks.Deploy.PackagePusher.Push(DownloadedPackage package, MachineEndpoint destination, ILog log)
at RedGate.Deploy.Server.Tasks.Deploy.UploadPackageActivity.<Execute>b__0()
at System.Threading.Tasks.Task.InnerInvoke()
at System.Threading.Tasks.Task.Execute()

Exception rethrown at [1]:
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccess(Task task)
at RedGate.Deploy.Server.Tasks.ActivityRuntime.<ExecuteChild>d__0.MoveNext()
2013-07-08 16:51:52 ERROR System.TimeoutException: The request channel timed out while waiting for a reply after 00:00:00. Increase the timeout value passed to the call to Request or increase the SendTimeout value on the Binding. The time allotted to this operation may have been a portion of a longer timeout. ---> System.TimeoutException: The request operation did not complete within the allotted timeout of 00:01:00. The time allotted to this operation may have been a portion of a longer timeout.
at System.ServiceModel.Channels.ReliableRequestSessionChannel.SyncRequest.WaitForReply(TimeSpan timeout)
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
--- End of inner exception stack trace ---

Server stack trace:
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at RedGate.Deploy.Shared.Contracts.IPackageService.FinalizeUpload()
at CallSite.Target(Closure , CallSite , Object )
at System.Dynamic.UpdateDelegates.UpdateAndExecuteVoid1[T0](CallSite site, T0 arg0)
at CallSite.Target(Closure , CallSite , Object )
at ImpromptuInterface.Optimization.InvokeHelper.InvokeMemberAction(CallSite& callsite, Type binderType, LazyBinder binder, String_OR_InvokeMemberName name, Boolean staticContext, Type context, String[] argNames, Object target, Object[] args)
at ImpromptuInterface.Optimization.InvokeHelper.InvokeMemberActionCallSite(Object target, String_OR_InvokeMemberName name, Object[] args, String[] tArgNames, Type tContext, Boolean tStaticContext, CallSite& callSite)
at ImpromptuInterface.Impromptu.InvokeMemberAction(Object target, String_OR_InvokeMemberName name, Object[] args)
at ImpromptuInterface.Dynamic.ImpromptuForwarder.TryInvokeMember(InvokeMemberBinder binder, Object[] args, Object& result)
at RedGate.Deploy.Server.Proxies.WcfSecurityWrapper`1.TryInvokeMember(InvokeMemberBinder binder, Object[] args, Object& result)
at CallSite.Target(Closure , CallSite , Object )
at ActLike_IPackageService_2d858676b1eb41aa9c0a21844e982fb8.FinalizeUpload()
at RedGate.Deploy.Server.Tasks.Deploy.PackagePusher.UploadFile(IPackageService service, String fullPath, ILog log)
at RedGate.Deploy.Server.Tasks.Deploy.PackagePusher.Push(DownloadedPackage package, MachineEndpoint destination, ILog log)
at RedGate.Deploy.Server.Tasks.Deploy.UploadPackageActivity.<Execute>b__0()
at System.Threading.Tasks.Task.InnerInvoke()
at System.Threading.Tasks.Task.Execute()

Exception rethrown at [1]:
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccess(Task task)
at RedGate.Deploy.Server.Tasks.ActivityRuntime.<ExecuteChild>d__0.MoveNext()
lee5i3
 
Posts: 38
Joined: Fri Jul 16, 2010 9:26 pm

Postby lee5i3 » Mon Jul 08, 2013 6:41 pm

I got it to work..

Had to keep trying a few times until it did not time out..

Anyway to increase the timeout so it doesn't happen again?
lee5i3
 
Posts: 38
Joined: Fri Jul 16, 2010 9:26 pm

Postby chirayu » Tue Jul 09, 2013 12:27 pm

Hi

Thanks for reporting this issue. We have entered a bug in our system (RGD-765) and are looking at this right now.

I will update you once we have decided on the fix for this.

Thanks!

Chirayu
Chirayu Shishodiya
Software Engineer - Deployment Manager
Red Gate
chirayu
 
Posts: 68
Joined: Mon Sep 17, 2012 5:48 pm

Postby lee5i3 » Tue Jul 09, 2013 3:42 pm

Thanks,

After trying to redeploy again, I am unable to deploy.
I tried half a dozen attempts, they all failed trying to upload packages.

I even tried unselected all except 1, but even though they are unselected, it still attempts to upload all packages.

Thankfully, our deployment schedule is every 3 weeks, so hopefully there will be a solution by then or I will need to deploy manually.
lee5i3
 
Posts: 38
Joined: Fri Jul 16, 2010 9:26 pm

Postby Emma A » Thu Jul 11, 2013 2:09 pm

This has been captured in an internal issue, bug number RGD-766 and has been addressed so should be available in the next release.
Thank you for the feedback.
Emma A
 
Posts: 40
Joined: Mon Mar 30, 2009 2:53 pm

Postby lee5i3 » Thu Jul 11, 2013 6:06 pm

Great!!

For now, I had to split up the services into multiple deployment manager projects, havent tried it yet, but hopefully if release isnt out before my next sprint, this should work..
lee5i3
 
Posts: 38
Joined: Fri Jul 16, 2010 9:26 pm

Postby Mike Upton » Tue Jul 16, 2013 4:53 pm

We've added a workaround for this problem (RGD-765) in the latest release (2.1.2.21). You can now configure the timeouts used in communications with a Deployment Agent. You need to edit the appSettings section of the Deployment Manager Server config file, C:\\Program Files (x86)\\Red Gate\\Deployment Manager\\Server\\RedGate.Deploy.Server.exe.config.

Add a new line inside the <appSettings> element that reads:
Code: Select all
<add key="Bindings.AgentSendTimeout" value="00:20:00" />

This will set the send timeout to 20 minutes, rather than the default 1 minute. Please note that this will also increase the time it takes for an Agent health check to fail if the Agent cannot be communicated with, which is why we don't use a very long timeout by default.

By the way, we've also fixed the bug where skipped steps would still have their packages sent to the agent, even though they weren't going to be deployed (bug reference RGD-766)

Hope that helps. Please let us know if you're still seeing timeout problems after changing that value.
Mike Upton

Software Engineer
Red Gate Software Ltd.
Mike Upton
 
Posts: 157
Joined: Wed May 11, 2011 8:04 am
Location: Red Gate

Postby lee5i3 » Tue Jul 16, 2013 8:04 pm

So far so good, successfully deployed 15 steps.. however that was localhost, real test will be on my next sprint release which is scheduled on the 29th..

I'll let you know how that goes
lee5i3
 
Posts: 38
Joined: Fri Jul 16, 2010 9:26 pm


Return to Deployment Manager

Who is online

Users browsing this forum: No registered users and 0 guests