Chromium Internals - Lifetime of a navigation: A closer look

, ,

Inspiration

This post is inspired from Nasko’s post Chromium Internals - Lifetime of a navigation and offers a deeper insight into the process of a navigation.

What’s a navigation?

It is not so easy to answer this question precisely since navigation is the core function of a browser. To have a better understanding of what a navigation is, let’s start from HTML5 specification.

From HTML5 specification

It’s astonishing for my first time to learn that there existed two HTML5 versions for a long time until May 2019. Briefly speaking, W3C expects the specification to be versioned periodically like HTML4.01 while WHATWG wants a much more flexible one called “Living Standard”. Currently, W3C and WHATWG works together on the Living Standard which I refer to in this post.

Many people would think the HTML5 specification as a standard for HTML syntax and features. That’s true indeed, though, things becomes much more complex than expected when it comes to details. For example, one possible value for the sandbox attribute of an iframe is allow-top-navigation, which makes it inevitable to define the navigation and many other concepts, like Element, Document and Window. It even contains many implementation details like event loop and task queue. In short, current HTML specification is more like a cookbook for building the whole web platform, not a simple language specification. It enhances the compatibility between browsers and users can share similar experience across different browsers.

The key concept related to our topic is browsing context, which is called a Frame in Chromium. Usually, a tab or iframe has a dedicated browsing context, which has a corresponding window and document. Roughly speaking, a navigation, or navigating a browsing context means switching its corresponding document to another one. Although it sounds very simple, in fact there are massive corner cases to handle and even the specification is not so perfect.

Chromium generally follows the navigation defined in the specification but classifies the navigation in a more proper way. Based on the destination document, the navigation is divided into same-document navigation and cross-document navigation. Most time, Chromium deals with cross-document navigation except scenarios below:

  • The URL is trying to navigate Chromium to a fragment in current page, like this.
  • The navigation is caused by History API.

On the other hand, based on the initiator, the navigation is also divided into browser-initiated navigation and renderer-initiated navigation. Note that here “browser” and “renderer” refer to the browser and renderer process. The browser process is the process which users interact with while the renderer process typically is responsible for parsing document, rendering page and executing user scripts. A detailed explanation is documented at here.

A browser-initiated navigation means the navigation is triggered from the browser process, like entering a URL in address bar and hitting enter while a renderer-initiated navigation is the opposite, like assigning a new location to window.location in JavaScript.

Next, we will inspect a navigation to https://example.com initiated from omnibox to show the lifetime of a navigation.

Lifetime of a navigation

OpenURL

The logic about the omnibox is located in components/omnibox/browser. It’s not so difficult to find that our target function is OmniboxEditModel::AcceptInput after reading some headers. It retrieves user input and finds a match, like matching “https://google.com“ from “https://goog“, “https://google.com/search?q=Lazymio“ from “Lazymio”. After that, it asks the omnibox controller to open such URL.

1
2
3
4
5
6
// omnibox_edit_model.cc:902
    controller_->OnAutocompleteAccept(
        match.destination_url, match.post_content.get(), disposition,
        ui::PageTransitionFromInt(match.transition |
                                  ui::PAGE_TRANSITION_FROM_ADDRESS_BAR),
        match.type, match_selection_timestamp);

Note that the user input is handled in UI thread and thus these functions are executed synchronously. Then, the omnibox controller asks the browser to open the URL.

1
2
3
// chrome_omnibox_edit_model.cc:31
  if (command_updater_)
    command_updater_->ExecuteCommand(IDC_OPEN_CURRENT_URL);
1
2
3
4
5
6
7
8
// browser_command_controller.cc:384
switch (id) {
    // ...
    case IDC_OPEN_CURRENT_URL:
      OpenCurrentURL(browser_);
      break;
    // ...
}

And the browser starts to load contents.

1
2
3
4
5
// browser_navigator.cc:650
        // Perform the actual navigation, tracking whether it came from the
        // renderer.

        LoadURLInContents(contents_to_navigate_or_insert, params->url, params);

Then the navigation controller involves, it determines the navigation type, handle a bunch of special URLs, finds the target frame which needs navigating and the most importantly, creates a corresponding NavigationRequest.

In fact, our navigation starts from here.

1
2
3
4
5
6
// navigation_controller_impl.cc:2928
  std::unique_ptr<NavigationRequest> request =
      CreateNavigationRequestFromLoadParams(
          node, params, override_user_agent, should_replace_current_entry,
          has_user_gesture, NavigationDownloadPolicy(), reload_type,
          pending_entry_, pending_entry_->GetFrameEntry(node));

A NavigationRequest tracks different stages of the navigation and contains the core logic of the navigation. The node which needs navigating will hold this NavigationRequest until it commits so the request will be bind to the node shortly after being created.

1
2
// frame_node_tree.cc:466
    navigation_request_ = std::move(navigation_request);

BeforeUnload

After the navigation starts, the very first thing to do is to unload current document, which usually fires beforeonload event. For example, when exiting a page with some drafts, the website may prompt you with Changes you made may not saved. Note that for same-document navigation, the beforeonload event won’t be fired and the browser will call NavigationRequest::BeginNavigation directly.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
// navigator_impl.cc:312
  bool should_dispatch_beforeunload =
      !NavigationTypeUtils::IsSameDocument(
          request->common_params().navigation_type) &&
      !request->common_params().is_history_navigation_in_new_child_frame &&
      frame_tree_node->current_frame_host()->ShouldDispatchBeforeUnload(
          false /* check_subframes_only */);

  // ...

  // Have the current renderer execute its beforeunload event if needed. If it
  // is not needed then NavigationRequest::BeginNavigation should be directly
  // called instead.
  if (should_dispatch_beforeunload) {
    frame_tree_node->navigation_request()->SetWaitingForRendererResponse();
    frame_tree_node->current_frame_host()->DispatchBeforeUnload(
        RenderFrameHostImpl::BeforeUnloadType::BROWSER_INITIATED_NAVIGATION,
        reload_type != ReloadType::NONE);
  } else {
    frame_tree_node->navigation_request()->BeginNavigation();
  }

To dispatch beforeunload, an IPC message is created and sent to the corresponding renderer. After the renderer finishes everything, it calls the browser to proceed the navigation by another IPC message.

1
2
3
4
5
6
7
8
9
10
11
12
// navigator_impl.cc:484
void NavigatorImpl::BeforeUnloadCompleted(FrameTreeNode* frame_tree_node,
                                          bool proceed,
                                          const base::TimeTicks& proceed_time) {
  // ...

  NavigationRequest* navigation_request = frame_tree_node->navigation_request();

  // ...
  
  navigation_request->BeginNavigation();
}

BeginNavigation

Then the browser calls NavigationRequest::BeginNavigation, but before a network request is built and sent, several navigation throttlers will decide whether the navigation should be proceeded, deferred or stopped.

1
2
3
4
// navigation_request.cc:3314
  // Notify each throttle of the request.
  throttle_runner_->ProcessNavigationEvent(
      NavigationThrottleRunner::Event::WillStartRequest);
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
//navigation_throttler_runner.cc:154
void NavigationThrottleRunner::ProcessInternal() {
  DCHECK_NE(Event::NoEvent, current_event_);
  base::WeakPtr<NavigationThrottleRunner> weak_ref = weak_factory_.GetWeakPtr();
  for (size_t i = next_index_; i < throttles_.size(); ++i) {
    TRACE_EVENT1("navigation", GetEventName(current_event_), "throttle",
                 throttles_[i]->GetNameForLogging());
    NavigationThrottle::ThrottleCheckResult result =
        ExecuteNavigationEvent(throttles_[i].get(), current_event_);
    if (!weak_ref) {
      // The NavigationThrottle execution has destroyed this
      // NavigationThrottleRunner. Return immediately.
      return;
    }
    TRACE_EVENT_ASYNC_STEP_INTO0(
        "navigation", "NavigationHandle", delegate_,
        base::StringPrintf("%s: %s: %d", GetEventName(current_event_),
                           throttles_[i]->GetNameForLogging(),
                           result.action()));
    switch (result.action()) {
      case NavigationThrottle::PROCEED:
        continue;

      case NavigationThrottle::BLOCK_REQUEST_AND_COLLAPSE:
      case NavigationThrottle::BLOCK_REQUEST:
      case NavigationThrottle::BLOCK_RESPONSE:
      case NavigationThrottle::CANCEL:
      case NavigationThrottle::CANCEL_AND_IGNORE:
        next_index_ = 0;
        InformDelegate(result);
        return;

      case NavigationThrottle::DEFER:
        next_index_ = i + 1;
        return;
    }
  }

  next_index_ = 0;
  InformDelegate(NavigationThrottle::PROCEED);
}

If the navigation passes through throttlers, it’s time to bind a renderer to the NavigationRequest and build a network request.

1
2
// navigation_request.cc:2231
  SetExpectedProcess(navigating_frame_host->GetProcess());
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
// navigation_request.cc:2340
  loader_ = NavigationURLLoader::Create(
      browser_context, partition,
      std::make_unique<NavigationRequestInfo>(
          common_params_->Clone(), begin_params_.Clone(), site_for_cookies,
          GetNetworkIsolationKey(), frame_tree_node_->IsMainFrame(),
          parent_is_main_frame, IsSecureFrame(frame_tree_node_->parent()),
          frame_tree_node_->frame_tree_node_id(),
          starting_site_instance_->IsGuest(), report_raw_headers,
          navigating_frame_host->GetVisibilityState() ==
              PageVisibilityState::kHiddenButPainting,
          upgrade_if_insecure_,
          blob_url_loader_factory_ ? blob_url_loader_factory_->Clone()
                                   : nullptr,
          devtools_navigation_token(), frame_tree_node_->devtools_frame_token(),
          OriginPolicyThrottle::ShouldRequestOriginPolicy(common_params_->url)),
      std::move(navigation_ui_data), service_worker_handle_.get(),
      appcache_handle_.get(), std::move(prefetched_signed_exchange_cache_),
      this, IsServedFromBackForwardCache(), std::move(interceptor));

After the URL loader loader_ is created, the network request will be sent and processed asynchronously. Note that the delegate pattern is applied in the implementation of NavigationURLLoader and the last but two argument this, NavigationRequest itself, is the delegate of the new loader_. Therefore, after the loader_ finishes loading the documents, it informs its delegate, the NavigationRequest by calling NavigationRequest::OnResponseStarted.

1
2
3
4
5
6
7
8
9
// navigation_url_loader_impl.cc:1471
  // TODO(scottmg): This needs to do more of what
  // NavigationResourceHandler::OnResponseStarted() does.
  delegate_->OnResponseStarted(
      std::move(url_loader_client_endpoints), std::move(response_head),
      std::move(response_body),
      GlobalRequestID(global_request_id.child_id, global_request_id.request_id),
      is_download, download_policy_,
      request_controller_->TakeSubresourceLoaderParams());

So this is the end of BeginNavigation phase. It seems to do lots of heavy work, though, almost every operation is asynchronous since it lives in UI thread.

CommitNavigation

When the response arrives (for simplicity, assume that there is no redirection), NavigationRequest::OnResponseStarted is called and the next thing is to commit this navigation. Similarly, before committing, several navigation throttlers decide the result of the navigation.

1
2
3
4
// navigation_request.cc:3369
  // Notify each throttle of the response.
  throttle_runner_->ProcessNavigationEvent(
      NavigationThrottleRunner::Event::WillProcessResponse);

And finally, after tons of security checks, in NavigationRequest::CommitNavigation, RenderFrameHostImpl::CommitNavigation is called to commit the navigation (with another tons of checks). And the browser will tell the corresponding renderer to commit the navigation by sending an IPC message.

1
2
3
4
5
6
7
8
9
// render_frame_host_impl.cc:5873
    SendCommitNavigation(
        navigation_client, navigation_request, std::move(common_params),
        std::move(commit_params), std::move(head), std::move(response_body),
        std::move(url_loader_client_endpoints),
        std::move(subresource_loader_factories),
        std::move(subresource_overrides), std::move(controller),
        std::move(provider_info), std::move(prefetch_loader_factory),
        devtools_navigation_token);

Now it’s time to focus on the renderer. After receiving the request to commit the navigation, the renderer firstly creates a body loader.

1
2
3
4
5
// navigation_body_loader.cc:85
    navigation_params->body_loader.reset(new NavigationBodyLoader(
        std::move(response_head), std::move(response_body),
        std::move(url_loader_client_endpoints), task_runner, render_frame_id,
        std::move(resource_load_info)));

Then it creates a document loader and commits it.

1
2
3
4
5
6
7
8
9
// frame_loader.cc:989
  DocumentLoader* new_document_loader = Client()->CreateDocumentLoader(
      frame_, navigation_type, content_security_policy,
      std::move(navigation_params), std::move(extra_data));

  CommitDocumentLoader(new_document_loader, unload_timing,
                       previous_history_item,
                       is_javascript_url ? CommitReason::kJavascriptUrl
                                         : CommitReason::kRegular);

Before the document loader really starts, it tells the browser process that it has finished committing the navigation.

1
2
3
4
5
6
7
8
9
// render_frame_impl.cc:4356
  DidCommitNavigationInternal(
      item, commit_type, false /* was_within_same_document */, transition,
      should_reset_browser_interface_broker
          ? mojom::DidCommitProvisionalLoadInterfaceParams::New(
                std::move(remote_interface_provider_receiver),
                std::move(browser_interface_broker_receiver))
          : nullptr,
      embedding_token);
1
2
3
4
// render_frame_impl.cc:5383
    if (navigation_state->uses_per_navigation_mojo_interface()) {
      navigation_state->RunPerNavigationInterfaceCommitNavigationCallback(
          std::move(params), std::move(interface_params));

And the document loader finally asks the body loader to load the body.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
// document_loader.cc:1320
  bool use_isolated_code_cache =
      RuntimeEnabledFeatures::CacheInlineScriptCodeEnabled() &&
      ShouldUseIsolatedCodeCache(mojom::RequestContextType::HYPERLINK,
                                 response_);

  // The |cached_metadata_handler_| is created, even when
  // |use_isolated_code_cache| is false to support the parts that don't
  // go throught the site-isolated-code-cache.
  auto cached_metadata_sender = CachedMetadataSender::Create(
      response_, blink::mojom::CodeCacheType::kJavascript, requestor_origin_);
  cached_metadata_handler_ =
      MakeGarbageCollected<SourceKeyedCachedMetadataHandler>(
          WTF::TextEncoding(), std::move(cached_metadata_sender));

  body_loader_->StartLoadingBody(this, use_isolated_code_cache);

After that, the renderer starts to parse HTML and build DOM tree asynchronously. Let’s go back to the browser process.

The first message to arrive is that the renderer has committed the navigation, which invokes DidCommitNavigation in various objects. The most important thing here is to update the origin of the navigated frame. In other words, if a navigation is cancelled before DidCommitNavigation, the origin of the frame remains correct at least, which ensures that same-origin policy won’t be compromised.

1
2
3
4
5
// navigator_impl.cc:184
  frame_tree_node->SetCurrentOrigin(
      params.origin, params.has_potentially_trustworthy_unique_origin);
  frame_tree_node->SetInsecureRequestPolicy(params.insecure_request_policy);
  frame_tree_node->SetInsecureNavigationsSet(params.insecure_navigations_set);

After DidCommitNavigation finishes, the navigation is done. From users’ view, almost nothing changes at this time, though, the browser has already committed the navigation and the next step is to load the document. After the renderer informs the frame that the loading is finished, the navigation (in a broad sense) finally ends.

Summary

After some reworks like PlzNavigate, the navigation design in Chromium is quite clear and decent. Below is a screenshot from Chrome University 2018: Life of a Navigation, a presentation made by Nasko, which illustrates how navigation works in a very high level.

Reference